2024-02-07T12:56:22.450 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-07T12:56:22.451 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548780 --description orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} --name gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi --owner scheduled_gabrioux@teuthology -- /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548780/orig.config.yaml 2024-02-07T12:56:22.480 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-07T12:56:22.549 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548780 branch: wip-guits-testing-4-2024-02-05-1854 description: orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} email: gabrioux@ibm.com first_in_suite: false job_id: '7548780' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 1 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-testing-4-2024-02-05-1854 ceph: conf: global: mon election default strategy: 1 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: true mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 44458ccedb18241cc2e17f10cdab3d1385399261 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 44458ccedb18241cc2e17f10cdab3d1385399261 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-guits-testing-4-2024-02-05-1854 sha1: 44458ccedb18241cc2e17f10cdab3d1385399261 owner: scheduled_gabrioux@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.1 - client.1 - prometheus.a - grafana.a - node-exporter.b - - mon.c - mgr.z - osd.2 - client.2 - node-exporter.c seed: 1701 sha1: 44458ccedb18241cc2e17f10cdab3d1385399261 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: wip-guits-testing-4-2024-02-05-1854 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_44458ccedb18241cc2e17f10cdab3d1385399261/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 44458ccedb18241cc2e17f10cdab3d1385399261 targets: smithi067.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBbRfxaR4TLBQUqZhdNEB6Zyt9efg308UYaaj47aM8raZaJe3NYGHgH0VB44jyLnElgoTBPRQUvpC9Qx6Vez9m0= smithi159.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHEw+djB8EJKcw0sOA9wPa/ZFIMr9kOVaW4Bfd9wkLaWufbicaEeT7nXXKB/tC91Vi7JM9NEIshAX0gmtK6WUKo= smithi191.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCgTAM8Rvcs7WzdlG747Si/SsWJQLrCEYd9wwNF1ludTA6s1fYN91CIQrbTwl2pneB4lCLe6eP72uSWvzz7sw1g= tasks: - pexec: all: - sudo dnf install runc -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - nvme_loop: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls --format yaml - ceph orch ls | grep '^osd ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: d9fdb2209e15b39d9f061fd85399f352ce0f0894 timestamp: 2024-02-06_08:19:47 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-07T12:56:22.550 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_44458ccedb18241cc2e17f10cdab3d1385399261/qa; will attempt to use it 2024-02-07T12:56:22.550 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_44458ccedb18241cc2e17f10cdab3d1385399261/qa/tasks 2024-02-07T12:56:22.550 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-07T12:56:22.552 INFO:teuthology.task.internal:Checking packages... 2024-02-07T12:56:22.575 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '44458ccedb18241cc2e17f10cdab3d1385399261' 2024-02-07T12:56:22.575 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-07T12:56:22.575 INFO:teuthology.packaging:ref: None 2024-02-07T12:56:22.575 INFO:teuthology.packaging:tag: None 2024-02-07T12:56:22.575 INFO:teuthology.packaging:branch: wip-guits-testing-4-2024-02-05-1854 2024-02-07T12:56:22.575 INFO:teuthology.packaging:sha1: 44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T12:56:22.575 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-testing-4-2024-02-05-1854 2024-02-07T12:56:22.711 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1252.g44458cce 2024-02-07T12:56:22.712 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-07T12:56:22.718 INFO:teuthology.task.internal:no buildpackages task found 2024-02-07T12:56:22.719 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-07T12:56:22.774 INFO:teuthology.task.internal:Saving configuration 2024-02-07T12:56:22.784 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-07T12:56:22.789 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-07T12:56:22.811 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548780', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-07 12:50:36.450458', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBbRfxaR4TLBQUqZhdNEB6Zyt9efg308UYaaj47aM8raZaJe3NYGHgH0VB44jyLnElgoTBPRQUvpC9Qx6Vez9m0='} 2024-02-07T12:56:22.832 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548780', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-07 12:50:36.448735', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHEw+djB8EJKcw0sOA9wPa/ZFIMr9kOVaW4Bfd9wkLaWufbicaEeT7nXXKB/tC91Vi7JM9NEIshAX0gmtK6WUKo='} 2024-02-07T12:56:22.853 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548780', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-07 12:50:36.451584', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCgTAM8Rvcs7WzdlG747Si/SsWJQLrCEYd9wwNF1ludTA6s1fYN91CIQrbTwl2pneB4lCLe6eP72uSWvzz7sw1g='} 2024-02-07T12:56:22.853 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-07T12:56:22.858 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-07T12:56:22.859 INFO:teuthology.task.internal:roles: ubuntu@smithi159.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.1', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2024-02-07T12:56:22.859 INFO:teuthology.task.internal:roles: ubuntu@smithi191.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'osd.2', 'client.2', 'node-exporter.c'] 2024-02-07T12:56:22.859 INFO:teuthology.run_tasks:Running task console_log... 2024-02-07T12:56:22.953 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fab26ad0f70>, signals=[15]) 2024-02-07T12:56:22.954 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-07T12:56:22.961 INFO:teuthology.task.internal:Opening connections... 2024-02-07T12:56:22.962 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi067.front.sepia.ceph.com 2024-02-07T12:56:22.964 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:56:23.069 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi159.front.sepia.ceph.com 2024-02-07T12:56:23.070 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:56:23.156 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi191.front.sepia.ceph.com 2024-02-07T12:56:23.157 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:56:23.243 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-07T12:56:23.249 DEBUG:teuthology.orchestra.run.smithi067:> uname -m 2024-02-07T12:56:23.268 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64 2024-02-07T12:56:23.268 DEBUG:teuthology.orchestra.run.smithi067:> cat /etc/os-release 2024-02-07T12:56:23.325 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="CentOS Stream" 2024-02-07T12:56:23.325 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="9" 2024-02-07T12:56:23.325 INFO:teuthology.orchestra.run.smithi067.stdout:ID="centos" 2024-02-07T12:56:23.325 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="rhel fedora" 2024-02-07T12:56:23.325 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="9" 2024-02-07T12:56:23.325 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el9" 2024-02-07T12:56:23.326 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-07T12:56:23.326 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31" 2024-02-07T12:56:23.326 INFO:teuthology.orchestra.run.smithi067.stdout:LOGO="fedora-logo-icon" 2024-02-07T12:56:23.326 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-07T12:56:23.326 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://centos.org/" 2024-02-07T12:56:23.326 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-07T12:56:23.326 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-07T12:56:23.326 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-07T12:56:23.327 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server 2024-02-07T12:56:23.347 DEBUG:teuthology.orchestra.run.smithi159:> uname -m 2024-02-07T12:56:23.365 INFO:teuthology.orchestra.run.smithi159.stdout:x86_64 2024-02-07T12:56:23.365 DEBUG:teuthology.orchestra.run.smithi159:> cat /etc/os-release 2024-02-07T12:56:23.422 INFO:teuthology.orchestra.run.smithi159.stdout:NAME="CentOS Stream" 2024-02-07T12:56:23.422 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION="9" 2024-02-07T12:56:23.422 INFO:teuthology.orchestra.run.smithi159.stdout:ID="centos" 2024-02-07T12:56:23.422 INFO:teuthology.orchestra.run.smithi159.stdout:ID_LIKE="rhel fedora" 2024-02-07T12:56:23.422 INFO:teuthology.orchestra.run.smithi159.stdout:VERSION_ID="9" 2024-02-07T12:56:23.422 INFO:teuthology.orchestra.run.smithi159.stdout:PLATFORM_ID="platform:el9" 2024-02-07T12:56:23.422 INFO:teuthology.orchestra.run.smithi159.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-07T12:56:23.422 INFO:teuthology.orchestra.run.smithi159.stdout:ANSI_COLOR="0;31" 2024-02-07T12:56:23.422 INFO:teuthology.orchestra.run.smithi159.stdout:LOGO="fedora-logo-icon" 2024-02-07T12:56:23.423 INFO:teuthology.orchestra.run.smithi159.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-07T12:56:23.423 INFO:teuthology.orchestra.run.smithi159.stdout:HOME_URL="https://centos.org/" 2024-02-07T12:56:23.423 INFO:teuthology.orchestra.run.smithi159.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-07T12:56:23.423 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-07T12:56:23.423 INFO:teuthology.orchestra.run.smithi159.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-07T12:56:23.423 INFO:teuthology.lock.ops:Updating smithi159.front.sepia.ceph.com on lock server 2024-02-07T12:56:23.448 DEBUG:teuthology.orchestra.run.smithi191:> uname -m 2024-02-07T12:56:23.465 INFO:teuthology.orchestra.run.smithi191.stdout:x86_64 2024-02-07T12:56:23.465 DEBUG:teuthology.orchestra.run.smithi191:> cat /etc/os-release 2024-02-07T12:56:23.522 INFO:teuthology.orchestra.run.smithi191.stdout:NAME="CentOS Stream" 2024-02-07T12:56:23.522 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION="9" 2024-02-07T12:56:23.522 INFO:teuthology.orchestra.run.smithi191.stdout:ID="centos" 2024-02-07T12:56:23.522 INFO:teuthology.orchestra.run.smithi191.stdout:ID_LIKE="rhel fedora" 2024-02-07T12:56:23.522 INFO:teuthology.orchestra.run.smithi191.stdout:VERSION_ID="9" 2024-02-07T12:56:23.522 INFO:teuthology.orchestra.run.smithi191.stdout:PLATFORM_ID="platform:el9" 2024-02-07T12:56:23.522 INFO:teuthology.orchestra.run.smithi191.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-07T12:56:23.523 INFO:teuthology.orchestra.run.smithi191.stdout:ANSI_COLOR="0;31" 2024-02-07T12:56:23.523 INFO:teuthology.orchestra.run.smithi191.stdout:LOGO="fedora-logo-icon" 2024-02-07T12:56:23.523 INFO:teuthology.orchestra.run.smithi191.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-07T12:56:23.523 INFO:teuthology.orchestra.run.smithi191.stdout:HOME_URL="https://centos.org/" 2024-02-07T12:56:23.523 INFO:teuthology.orchestra.run.smithi191.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-07T12:56:23.523 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-07T12:56:23.523 INFO:teuthology.orchestra.run.smithi191.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-07T12:56:23.523 INFO:teuthology.lock.ops:Updating smithi191.front.sepia.ceph.com on lock server 2024-02-07T12:56:23.547 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-07T12:56:23.556 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-07T12:56:23.570 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-07T12:56:23.570 DEBUG:teuthology.orchestra.run.smithi067:> test '!' -e /home/ubuntu/cephtest 2024-02-07T12:56:23.573 DEBUG:teuthology.orchestra.run.smithi159:> test '!' -e /home/ubuntu/cephtest 2024-02-07T12:56:23.575 DEBUG:teuthology.orchestra.run.smithi191:> test '!' -e /home/ubuntu/cephtest 2024-02-07T12:56:23.591 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-07T12:56:23.597 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-07T12:56:23.597 DEBUG:teuthology.orchestra.run.smithi067:> test -z $(ls -A /var/lib/ceph) 2024-02-07T12:56:23.630 DEBUG:teuthology.orchestra.run.smithi159:> test -z $(ls -A /var/lib/ceph) 2024-02-07T12:56:23.632 DEBUG:teuthology.orchestra.run.smithi191:> test -z $(ls -A /var/lib/ceph) 2024-02-07T12:56:23.745 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-07T12:56:23.817 INFO:teuthology.run_tasks:Running task kernel... 2024-02-07T12:56:23.830 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-07T12:56:23.830 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-07T12:56:23.831 DEBUG:teuthology.orchestra.run.smithi067:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-07T12:56:23.831 DEBUG:teuthology.orchestra.run.smithi159:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-07T12:56:23.832 DEBUG:teuthology.orchestra.run.smithi191:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-07T12:56:23.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:56:23.848 DEBUG:teuthology.orchestra.run.smithi159:> uname -r 2024-02-07T12:56:23.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:56:23.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T12:56:23.849 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-02-07T12:56:23.850 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2024-02-07T12:56:23.905 INFO:teuthology.orchestra.run.smithi159.stdout:5.14.0-361.el9.x86_64 2024-02-07T12:56:23.905 INFO:teuthology.task.kernel:Running kernel on smithi159: 5.14.0-361.el9.x86_64 2024-02-07T12:56:23.905 DEBUG:teuthology.orchestra.run.smithi159:> sudo yum install -y kernel 2024-02-07T12:56:23.906 INFO:teuthology.orchestra.run.smithi191.stdout:5.14.0-361.el9.x86_64 2024-02-07T12:56:23.907 INFO:teuthology.task.kernel:Running kernel on smithi191: 5.14.0-361.el9.x86_64 2024-02-07T12:56:23.907 DEBUG:teuthology.orchestra.run.smithi191:> sudo yum install -y kernel 2024-02-07T12:56:23.908 INFO:teuthology.orchestra.run.smithi067.stdout:5.14.0-361.el9.x86_64 2024-02-07T12:56:23.908 INFO:teuthology.task.kernel:Running kernel on smithi067: 5.14.0-361.el9.x86_64 2024-02-07T12:56:23.908 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum install -y kernel 2024-02-07T12:56:25.896 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-9 - AppStream 18 MB/s | 8.0 MB 00:00 2024-02-07T12:56:26.148 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS-9 - AppStream 14 MB/s | 8.0 MB 00:00 2024-02-07T12:56:26.265 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-9 - AppStream 11 MB/s | 8.0 MB 00:00 2024-02-07T12:56:27.730 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS-9 - BaseOS 5.2 MB/s | 2.5 MB 00:00 2024-02-07T12:56:27.966 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS-9 - BaseOS 4.7 MB/s | 2.5 MB 00:00 2024-02-07T12:56:28.065 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS-9 - BaseOS 5.1 MB/s | 2.5 MB 00:00 2024-02-07T12:56:29.115 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-02-07T12:56:30.131 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - BaseOS 3.9 MB/s | 8.0 MB 00:02 2024-02-07T12:56:31.655 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.0 MB 00:03 2024-02-07T12:56:33.165 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - AppStream 6.4 MB/s | 19 MB 00:02 2024-02-07T12:56:34.042 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - AppStream 15 MB/s | 19 MB 00:01 2024-02-07T12:56:34.355 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 19 MB 00:03 2024-02-07T12:56:37.358 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - CRB 9.8 MB/s | 5.9 MB 00:00 2024-02-07T12:56:38.074 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - CRB 13 MB/s | 5.9 MB 00:00 2024-02-07T12:56:38.943 INFO:teuthology.orchestra.run.smithi191.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 15 kB 00:00 2024-02-07T12:56:39.447 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - CRB 4.4 MB/s | 5.9 MB 00:01 2024-02-07T12:56:39.917 INFO:teuthology.orchestra.run.smithi159.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 15 kB 00:00 2024-02-07T12:56:40.378 INFO:teuthology.orchestra.run.smithi191.stdout:Extra Packages for Enterprise Linux 16 MB/s | 20 MB 00:01 2024-02-07T12:56:40.889 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 15 kB 00:00 2024-02-07T12:56:45.309 INFO:teuthology.orchestra.run.smithi191.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-02-07T12:56:47.518 INFO:teuthology.orchestra.run.smithi191.stderr:Modular dependency problems: 2024-02-07T12:56:47.518 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-02-07T12:56:47.518 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 1: conflicting requests 2024-02-07T12:56:47.518 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240206092359:20125149.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.518 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 2: conflicting requests 2024-02-07T12:56:47.518 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.518 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 3: conflicting requests 2024-02-07T12:56:47.518 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.518 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 4: conflicting requests 2024-02-07T12:56:47.518 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.518 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 5: conflicting requests 2024-02-07T12:56:47.519 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820231031173453:b754926a.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.519 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 6: conflicting requests 2024-02-07T12:56:47.519 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820230817140043:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.519 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 7: conflicting requests 2024-02-07T12:56:47.519 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.519 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 8: conflicting requests 2024-02-07T12:56:47.519 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.519 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 9: conflicting requests 2024-02-07T12:56:47.519 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.519 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 10: conflicting requests 2024-02-07T12:56:47.519 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.519 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 11: conflicting requests 2024-02-07T12:56:47.519 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.520 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 12: conflicting requests 2024-02-07T12:56:47.520 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.520 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 13: conflicting requests 2024-02-07T12:56:47.520 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.520 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 14: conflicting requests 2024-02-07T12:56:47.520 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.520 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 15: conflicting requests 2024-02-07T12:56:47.520 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.520 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 16: conflicting requests 2024-02-07T12:56:47.520 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.520 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 17: conflicting requests 2024-02-07T12:56:47.520 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.520 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 18: conflicting requests 2024-02-07T12:56:47.521 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.521 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 19: conflicting requests 2024-02-07T12:56:47.521 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.521 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 20: conflicting requests 2024-02-07T12:56:47.521 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.521 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 21: conflicting requests 2024-02-07T12:56:47.521 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.521 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 22: conflicting requests 2024-02-07T12:56:47.521 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.521 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 23: conflicting requests 2024-02-07T12:56:47.521 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240202092102:182f7c73.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.522 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 24: conflicting requests 2024-02-07T12:56:47.522 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.522 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 25: conflicting requests 2024-02-07T12:56:47.522 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.522 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 26: conflicting requests 2024-02-07T12:56:47.522 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240117165712:17377f89.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.522 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 27: conflicting requests 2024-02-07T12:56:47.522 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.522 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 28: conflicting requests 2024-02-07T12:56:47.522 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.522 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 29: conflicting requests 2024-02-07T12:56:47.522 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.522 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 30: conflicting requests 2024-02-07T12:56:47.522 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.523 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 31: conflicting requests 2024-02-07T12:56:47.523 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.523 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 32: conflicting requests 2024-02-07T12:56:47.523 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module squid:4:820230817231530:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.523 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 33: conflicting requests 2024-02-07T12:56:47.523 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.523 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 34: conflicting requests 2024-02-07T12:56:47.523 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.523 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 35: conflicting requests 2024-02-07T12:56:47.523 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.523 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 36: conflicting requests 2024-02-07T12:56:47.523 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.524 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 37: conflicting requests 2024-02-07T12:56:47.524 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.524 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.524 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.524 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 38: cannot install the best candidate for the job 2024-02-07T12:56:47.524 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.524 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.524 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.524 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.524 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 39: conflicting requests 2024-02-07T12:56:47.524 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.524 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.525 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.525 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.525 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.525 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.525 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 40: cannot install the best candidate for the job 2024-02-07T12:56:47.525 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.525 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.525 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.525 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.525 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 41: cannot install the best candidate for the job 2024-02-07T12:56:47.525 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.526 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.526 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.526 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.526 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 42: cannot install the best candidate for the job 2024-02-07T12:56:47.526 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.526 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.526 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.526 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.526 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 43: cannot install the best candidate for the job 2024-02-07T12:56:47.526 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.526 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.526 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.527 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.527 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 44: cannot install the best candidate for the job 2024-02-07T12:56:47.527 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.527 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.527 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.527 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.527 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 45: cannot install the best candidate for the job 2024-02-07T12:56:47.527 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.527 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.527 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.527 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-07T12:56:47.950 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-07T12:56:47.950 INFO:teuthology.orchestra.run.smithi191.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-07T12:56:48.075 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-07T12:56:48.080 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-07T12:56:48.080 INFO:teuthology.orchestra.run.smithi191.stdout: Package Architecture Version Repository Size 2024-02-07T12:56:48.080 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-07T12:56:48.080 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-07T12:56:48.080 INFO:teuthology.orchestra.run.smithi191.stdout: kernel x86_64 5.14.0-412.el9 baseos 6.0 M 2024-02-07T12:56:48.080 INFO:teuthology.orchestra.run.smithi191.stdout:Installing dependencies: 2024-02-07T12:56:48.080 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-core x86_64 5.14.0-412.el9 baseos 21 M 2024-02-07T12:56:48.080 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules x86_64 5.14.0-412.el9 baseos 39 M 2024-02-07T12:56:48.080 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules-core x86_64 5.14.0-412.el9 baseos 33 M 2024-02-07T12:56:48.080 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-07T12:56:48.080 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-07T12:56:48.080 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-07T12:56:48.081 INFO:teuthology.orchestra.run.smithi191.stdout:Install 4 Packages 2024-02-07T12:56:48.081 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-07T12:56:48.081 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 99 M 2024-02-07T12:56:48.081 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 122 M 2024-02-07T12:56:48.081 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-07T12:56:53.660 INFO:teuthology.orchestra.run.smithi191.stdout:(1/4): kernel-5.14.0-412.el9.x86_64.rpm 1.1 MB/s | 6.0 MB 00:05 2024-02-07T12:56:56.368 INFO:teuthology.orchestra.run.smithi191.stdout:(2/4): kernel-core-5.14.0-412.el9.x86_64.rpm 2.6 MB/s | 21 MB 00:08 2024-02-07T12:56:58.651 INFO:teuthology.orchestra.run.smithi191.stdout:(3/4): kernel-modules-core-5.14.0-412.el9.x86_6 6.7 MB/s | 33 MB 00:04 2024-02-07T12:57:00.394 INFO:teuthology.orchestra.run.smithi191.stdout:(4/4): kernel-modules-5.14.0-412.el9.x86_64.rpm 3.2 MB/s | 39 MB 00:12 2024-02-07T12:57:00.394 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-07T12:57:00.394 INFO:teuthology.orchestra.run.smithi191.stdout:Total 8.1 MB/s | 99 MB 00:12 2024-02-07T12:57:01.275 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-07T12:57:01.352 INFO:teuthology.orchestra.run.smithi159.stdout:Extra Packages for Enterprise Linux 978 kB/s | 20 MB 00:21 2024-02-07T12:57:01.647 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-07T12:57:01.647 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-07T12:57:02.184 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 984 kB/s | 20 MB 00:21 2024-02-07T12:57:03.001 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-07T12:57:03.001 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-07T12:57:05.967 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-07T12:57:06.548 INFO:teuthology.orchestra.run.smithi159.stdout:lab-extras 4.8 kB/s | 1.7 kB 00:00 2024-02-07T12:57:06.620 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-modules-core-5.14.0-412.el9.x86_64 1/4 2024-02-07T12:57:06.745 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-core-5.14.0-412.el9.x86_64 2/4 2024-02-07T12:57:07.272 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-02-07T12:57:08.756 INFO:teuthology.orchestra.run.smithi159.stderr:Modular dependency problems: 2024-02-07T12:57:08.756 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-02-07T12:57:08.756 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 1: conflicting requests 2024-02-07T12:57:08.756 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240206092359:20125149.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.756 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 2: conflicting requests 2024-02-07T12:57:08.756 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.756 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 3: conflicting requests 2024-02-07T12:57:08.756 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.756 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 4: conflicting requests 2024-02-07T12:57:08.757 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.757 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 5: conflicting requests 2024-02-07T12:57:08.757 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820231031173453:b754926a.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.757 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 6: conflicting requests 2024-02-07T12:57:08.757 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820230817140043:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.757 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 7: conflicting requests 2024-02-07T12:57:08.757 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.757 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 8: conflicting requests 2024-02-07T12:57:08.757 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.757 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 9: conflicting requests 2024-02-07T12:57:08.757 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.757 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 10: conflicting requests 2024-02-07T12:57:08.757 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.757 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 11: conflicting requests 2024-02-07T12:57:08.757 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.758 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 12: conflicting requests 2024-02-07T12:57:08.758 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.758 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 13: conflicting requests 2024-02-07T12:57:08.758 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.758 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 14: conflicting requests 2024-02-07T12:57:08.758 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.758 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 15: conflicting requests 2024-02-07T12:57:08.758 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.758 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 16: conflicting requests 2024-02-07T12:57:08.758 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.758 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 17: conflicting requests 2024-02-07T12:57:08.758 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.758 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 18: conflicting requests 2024-02-07T12:57:08.758 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.758 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 19: conflicting requests 2024-02-07T12:57:08.759 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.759 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 20: conflicting requests 2024-02-07T12:57:08.759 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.759 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 21: conflicting requests 2024-02-07T12:57:08.759 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.759 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 22: conflicting requests 2024-02-07T12:57:08.759 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.759 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 23: conflicting requests 2024-02-07T12:57:08.759 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240202092102:182f7c73.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.759 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 24: conflicting requests 2024-02-07T12:57:08.759 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.759 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 25: conflicting requests 2024-02-07T12:57:08.759 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.759 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 26: conflicting requests 2024-02-07T12:57:08.759 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240117165712:17377f89.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.760 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 27: conflicting requests 2024-02-07T12:57:08.760 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.760 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 28: conflicting requests 2024-02-07T12:57:08.760 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.760 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 29: conflicting requests 2024-02-07T12:57:08.760 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.760 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 30: conflicting requests 2024-02-07T12:57:08.760 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.760 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 31: conflicting requests 2024-02-07T12:57:08.760 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.760 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 32: conflicting requests 2024-02-07T12:57:08.760 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module squid:4:820230817231530:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.760 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 33: conflicting requests 2024-02-07T12:57:08.760 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.760 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 34: conflicting requests 2024-02-07T12:57:08.760 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.761 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 35: conflicting requests 2024-02-07T12:57:08.761 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.761 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 36: conflicting requests 2024-02-07T12:57:08.761 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.761 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 37: conflicting requests 2024-02-07T12:57:08.761 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.761 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.761 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.761 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 38: cannot install the best candidate for the job 2024-02-07T12:57:08.761 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.761 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.761 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.761 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.761 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 39: conflicting requests 2024-02-07T12:57:08.761 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.761 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.762 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.762 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.762 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.762 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.762 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 40: cannot install the best candidate for the job 2024-02-07T12:57:08.762 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.762 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.762 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.762 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.762 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 41: cannot install the best candidate for the job 2024-02-07T12:57:08.762 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.762 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.762 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.762 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.762 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 42: cannot install the best candidate for the job 2024-02-07T12:57:08.763 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.763 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.763 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.763 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.763 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 43: cannot install the best candidate for the job 2024-02-07T12:57:08.763 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.763 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.763 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.763 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.763 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 44: cannot install the best candidate for the job 2024-02-07T12:57:08.763 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.763 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.763 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.763 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.763 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 45: cannot install the best candidate for the job 2024-02-07T12:57:08.764 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.764 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.764 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-07T12:57:08.764 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.189 INFO:teuthology.orchestra.run.smithi159.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-07T12:57:09.189 INFO:teuthology.orchestra.run.smithi159.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-07T12:57:09.295 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-02-07T12:57:09.300 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-02-07T12:57:09.300 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2024-02-07T12:57:09.300 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-02-07T12:57:09.301 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-02-07T12:57:09.301 INFO:teuthology.orchestra.run.smithi159.stdout: kernel x86_64 5.14.0-412.el9 baseos 6.0 M 2024-02-07T12:57:09.301 INFO:teuthology.orchestra.run.smithi159.stdout:Installing dependencies: 2024-02-07T12:57:09.301 INFO:teuthology.orchestra.run.smithi159.stdout: kernel-core x86_64 5.14.0-412.el9 baseos 21 M 2024-02-07T12:57:09.301 INFO:teuthology.orchestra.run.smithi159.stdout: kernel-modules x86_64 5.14.0-412.el9 baseos 39 M 2024-02-07T12:57:09.301 INFO:teuthology.orchestra.run.smithi159.stdout: kernel-modules-core x86_64 5.14.0-412.el9 baseos 33 M 2024-02-07T12:57:09.301 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-02-07T12:57:09.301 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-02-07T12:57:09.301 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-02-07T12:57:09.301 INFO:teuthology.orchestra.run.smithi159.stdout:Install 4 Packages 2024-02-07T12:57:09.301 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-02-07T12:57:09.302 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 99 M 2024-02-07T12:57:09.302 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 122 M 2024-02-07T12:57:09.302 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-02-07T12:57:09.534 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-core-5.14.0-412.el9.x86_64 2/4 2024-02-07T12:57:09.584 INFO:teuthology.orchestra.run.smithi067.stderr:Modular dependency problems: 2024-02-07T12:57:09.584 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-07T12:57:09.584 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 1: conflicting requests 2024-02-07T12:57:09.584 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240206092359:20125149.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.584 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 2: conflicting requests 2024-02-07T12:57:09.585 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.585 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 3: conflicting requests 2024-02-07T12:57:09.585 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.585 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 4: conflicting requests 2024-02-07T12:57:09.585 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.585 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 5: conflicting requests 2024-02-07T12:57:09.585 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820231031173453:b754926a.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.585 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 6: conflicting requests 2024-02-07T12:57:09.585 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820230817140043:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.585 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 7: conflicting requests 2024-02-07T12:57:09.585 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.585 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 8: conflicting requests 2024-02-07T12:57:09.585 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.585 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 9: conflicting requests 2024-02-07T12:57:09.585 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.585 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 10: conflicting requests 2024-02-07T12:57:09.586 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.586 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 11: conflicting requests 2024-02-07T12:57:09.586 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.586 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 12: conflicting requests 2024-02-07T12:57:09.586 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.586 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 13: conflicting requests 2024-02-07T12:57:09.586 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.586 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 14: conflicting requests 2024-02-07T12:57:09.586 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.586 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 15: conflicting requests 2024-02-07T12:57:09.586 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.586 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 16: conflicting requests 2024-02-07T12:57:09.586 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.586 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 17: conflicting requests 2024-02-07T12:57:09.586 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.587 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 18: conflicting requests 2024-02-07T12:57:09.587 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.587 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 19: conflicting requests 2024-02-07T12:57:09.587 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.587 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 20: conflicting requests 2024-02-07T12:57:09.587 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.587 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 21: conflicting requests 2024-02-07T12:57:09.587 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.587 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 22: conflicting requests 2024-02-07T12:57:09.587 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.587 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 23: conflicting requests 2024-02-07T12:57:09.587 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240202092102:182f7c73.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.587 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 24: conflicting requests 2024-02-07T12:57:09.587 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.587 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 25: conflicting requests 2024-02-07T12:57:09.588 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.588 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 26: conflicting requests 2024-02-07T12:57:09.588 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240117165712:17377f89.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.588 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 27: conflicting requests 2024-02-07T12:57:09.588 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.588 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 28: conflicting requests 2024-02-07T12:57:09.588 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.588 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 29: conflicting requests 2024-02-07T12:57:09.588 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.588 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 30: conflicting requests 2024-02-07T12:57:09.588 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.588 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 31: conflicting requests 2024-02-07T12:57:09.588 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.588 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 32: conflicting requests 2024-02-07T12:57:09.588 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module squid:4:820230817231530:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.589 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 33: conflicting requests 2024-02-07T12:57:09.589 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.589 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 34: conflicting requests 2024-02-07T12:57:09.589 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.589 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 35: conflicting requests 2024-02-07T12:57:09.589 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.589 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 36: conflicting requests 2024-02-07T12:57:09.589 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.589 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 37: conflicting requests 2024-02-07T12:57:09.589 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.589 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.589 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.589 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 38: cannot install the best candidate for the job 2024-02-07T12:57:09.589 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.589 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.589 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.590 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.590 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 39: conflicting requests 2024-02-07T12:57:09.590 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.590 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.590 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.590 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.590 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.590 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.590 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 40: cannot install the best candidate for the job 2024-02-07T12:57:09.590 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.590 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.590 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.590 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.590 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 41: cannot install the best candidate for the job 2024-02-07T12:57:09.591 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.591 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.591 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.591 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.591 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 42: cannot install the best candidate for the job 2024-02-07T12:57:09.591 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.591 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.591 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.591 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.591 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 43: cannot install the best candidate for the job 2024-02-07T12:57:09.591 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.591 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.591 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.591 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.591 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 44: cannot install the best candidate for the job 2024-02-07T12:57:09.591 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.592 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.592 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.592 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.592 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 45: cannot install the best candidate for the job 2024-02-07T12:57:09.592 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.592 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.592 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.592 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-07T12:57:09.723 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-modules-5.14.0-412.el9.x86_64 3/4 2024-02-07T12:57:10.188 INFO:teuthology.orchestra.run.smithi067.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-07T12:57:10.188 INFO:teuthology.orchestra.run.smithi067.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-07T12:57:10.293 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-02-07T12:57:10.301 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-02-07T12:57:10.301 INFO:teuthology.orchestra.run.smithi067.stdout: Package Architecture Version Repository Size 2024-02-07T12:57:10.301 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-02-07T12:57:10.301 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-02-07T12:57:10.301 INFO:teuthology.orchestra.run.smithi067.stdout: kernel x86_64 5.14.0-412.el9 baseos 6.0 M 2024-02-07T12:57:10.301 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-02-07T12:57:10.302 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-core x86_64 5.14.0-412.el9 baseos 21 M 2024-02-07T12:57:10.302 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules x86_64 5.14.0-412.el9 baseos 39 M 2024-02-07T12:57:10.302 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules-core x86_64 5.14.0-412.el9 baseos 33 M 2024-02-07T12:57:10.302 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T12:57:10.302 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-02-07T12:57:10.302 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-02-07T12:57:10.302 INFO:teuthology.orchestra.run.smithi067.stdout:Install 4 Packages 2024-02-07T12:57:10.302 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T12:57:10.303 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 99 M 2024-02-07T12:57:10.303 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 122 M 2024-02-07T12:57:10.303 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-02-07T12:57:10.375 INFO:teuthology.orchestra.run.smithi159.stdout:(1/4): kernel-5.14.0-412.el9.x86_64.rpm 6.0 MB/s | 6.0 MB 00:01 2024-02-07T12:57:11.215 INFO:teuthology.orchestra.run.smithi067.stdout:(1/4): kernel-5.14.0-412.el9.x86_64.rpm 6.9 MB/s | 6.0 MB 00:00 2024-02-07T12:57:11.725 INFO:teuthology.orchestra.run.smithi159.stdout:(2/4): kernel-core-5.14.0-412.el9.x86_64.rpm 8.8 MB/s | 21 MB 00:02 2024-02-07T12:57:13.575 INFO:teuthology.orchestra.run.smithi159.stdout:(3/4): kernel-modules-5.14.0-412.el9.x86_64.rpm 9.3 MB/s | 39 MB 00:04 2024-02-07T12:57:14.025 INFO:teuthology.orchestra.run.smithi159.stdout:(4/4): kernel-modules-core-5.14.0-412.el9.x86_6 9.1 MB/s | 33 MB 00:03 2024-02-07T12:57:14.025 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-02-07T12:57:14.026 INFO:teuthology.orchestra.run.smithi159.stdout:Total 21 MB/s | 99 MB 00:04 2024-02-07T12:57:14.413 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-modules-5.14.0-412.el9.x86_64 3/4 2024-02-07T12:57:14.533 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : kernel-5.14.0-412.el9.x86_64 4/4 2024-02-07T12:57:14.902 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-02-07T12:57:15.322 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-02-07T12:57:15.322 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-02-07T12:57:15.432 INFO:teuthology.orchestra.run.smithi067.stdout:(2/4): kernel-core-5.14.0-412.el9.x86_64.rpm 4.1 MB/s | 21 MB 00:05 2024-02-07T12:57:15.890 INFO:teuthology.orchestra.run.smithi067.stdout:(3/4): kernel-modules-core-5.14.0-412.el9.x86_6 7.1 MB/s | 33 MB 00:04 2024-02-07T12:57:16.987 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-02-07T12:57:16.988 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-02-07T12:57:18.441 INFO:teuthology.orchestra.run.smithi067.stdout:(4/4): kernel-modules-5.14.0-412.el9.x86_64.rpm 4.8 MB/s | 39 MB 00:08 2024-02-07T12:57:18.441 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-02-07T12:57:18.441 INFO:teuthology.orchestra.run.smithi067.stdout:Total 12 MB/s | 99 MB 00:08 2024-02-07T12:57:19.233 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-modules-core-5.14.0-412.el9.x86_64 4/4 2024-02-07T12:57:19.359 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-02-07T12:57:19.705 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-02-07T12:57:19.793 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-02-07T12:57:19.793 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-02-07T12:57:20.376 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : kernel-modules-core-5.14.0-412.el9.x86_64 1/4 2024-02-07T12:57:20.509 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : kernel-core-5.14.0-412.el9.x86_64 2/4 2024-02-07T12:57:21.218 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-02-07T12:57:21.218 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-02-07T12:57:23.297 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: kernel-core-5.14.0-412.el9.x86_64 2/4 2024-02-07T12:57:23.487 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : kernel-modules-5.14.0-412.el9.x86_64 3/4 2024-02-07T12:57:24.053 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-02-07T12:57:24.710 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-modules-core-5.14.0-412.el9.x86_64 1/4 2024-02-07T12:57:24.841 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-core-5.14.0-412.el9.x86_64 2/4 2024-02-07T12:57:27.725 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-core-5.14.0-412.el9.x86_64 2/4 2024-02-07T12:57:27.909 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-modules-5.14.0-412.el9.x86_64 3/4 2024-02-07T12:57:28.165 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: kernel-modules-5.14.0-412.el9.x86_64 3/4 2024-02-07T12:57:28.288 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : kernel-5.14.0-412.el9.x86_64 4/4 2024-02-07T12:57:32.978 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-5.14.0-412.el9.x86_64 3/4 2024-02-07T12:57:33.048 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: kernel-modules-core-5.14.0-412.el9.x86_64 4/4 2024-02-07T12:57:33.087 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-5.14.0-412.el9.x86_64 4/4 2024-02-07T12:57:38.186 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-core-5.14.0-412.el9.x86_64 4/4 2024-02-07T12:57:48.444 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-core-5.14.0-412.el9.x86_64 4/4 2024-02-07T12:57:48.445 INFO:teuthology.orchestra.run.smithi191.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-07T12:57:48.445 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-07T12:57:48.530 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-modules-5.14.0-412.el9.x86_64 4/4 2024-02-07T12:57:48.992 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: kernel-5.14.0-412.el9.x86_64 4/4 2024-02-07T12:57:48.992 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-5.14.0-412.el9.x86_64 1/4 2024-02-07T12:57:48.992 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-core-5.14.0-412.el9.x86_64 2/4 2024-02-07T12:57:48.992 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-modules-5.14.0-412.el9.x86_64 3/4 2024-02-07T12:57:49.446 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : kernel-modules-core-5.14.0-412.el9.x86_64 4/4 2024-02-07T12:57:49.447 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-07T12:57:49.447 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-07T12:57:49.447 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-5.14.0-412.el9.x86_64 2024-02-07T12:57:49.447 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-core-5.14.0-412.el9.x86_64 2024-02-07T12:57:49.447 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules-5.14.0-412.el9.x86_64 2024-02-07T12:57:49.447 INFO:teuthology.orchestra.run.smithi191.stdout: kernel-modules-core-5.14.0-412.el9.x86_64 2024-02-07T12:57:49.447 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-07T12:57:49.447 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-07T12:57:49.986 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q kernel | sort -rV | head -n 1 2024-02-07T12:57:50.166 INFO:teuthology.orchestra.run.smithi191.stdout:kernel-5.14.0-412.el9.x86_64 2024-02-07T12:57:50.167 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-412.el9.x86_64 2024-02-07T12:57:50.167 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-412.el9.x86_64 2024-02-07T12:57:50.167 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-07T12:57:50.167 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-07T12:57:50.167 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-02-07T12:57:50.167 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi191.front.sepia.ceph.com, path=None, version=distro) 2024-02-07T12:57:50.167 DEBUG:teuthology.orchestra.run.smithi191:> rpm -q kernel | sort -rV | head -n 1 2024-02-07T12:57:50.289 INFO:teuthology.orchestra.run.smithi191.stdout:kernel-5.14.0-412.el9.x86_64 2024-02-07T12:57:50.289 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-412.el9.x86_64 2024-02-07T12:57:50.289 DEBUG:teuthology.orchestra.run.smithi191:> sudo rpm -qi grub2-tools 2024-02-07T12:57:50.323 INFO:teuthology.orchestra.run.smithi191.stdout:Name : grub2-tools 2024-02-07T12:57:50.323 INFO:teuthology.orchestra.run.smithi191.stdout:Epoch : 1 2024-02-07T12:57:50.323 INFO:teuthology.orchestra.run.smithi191.stdout:Version : 2.06 2024-02-07T12:57:50.323 INFO:teuthology.orchestra.run.smithi191.stdout:Release : 68.el9 2024-02-07T12:57:50.323 INFO:teuthology.orchestra.run.smithi191.stdout:Architecture: x86_64 2024-02-07T12:57:50.323 INFO:teuthology.orchestra.run.smithi191.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-07T12:57:50.324 INFO:teuthology.orchestra.run.smithi191.stdout:Group : Unspecified 2024-02-07T12:57:50.324 INFO:teuthology.orchestra.run.smithi191.stdout:Size : 8264254 2024-02-07T12:57:50.324 INFO:teuthology.orchestra.run.smithi191.stdout:License : GPLv3+ 2024-02-07T12:57:50.324 INFO:teuthology.orchestra.run.smithi191.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-07T12:57:50.324 INFO:teuthology.orchestra.run.smithi191.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-07T12:57:50.324 INFO:teuthology.orchestra.run.smithi191.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-07T12:57:50.324 INFO:teuthology.orchestra.run.smithi191.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-07T12:57:50.324 INFO:teuthology.orchestra.run.smithi191.stdout:Packager : builder@centos.org 2024-02-07T12:57:50.324 INFO:teuthology.orchestra.run.smithi191.stdout:Vendor : CentOS 2024-02-07T12:57:50.324 INFO:teuthology.orchestra.run.smithi191.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-07T12:57:50.324 INFO:teuthology.orchestra.run.smithi191.stdout:Summary : Support tools for GRUB. 2024-02-07T12:57:50.324 INFO:teuthology.orchestra.run.smithi191.stdout:Description : 2024-02-07T12:57:50.324 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-07T12:57:50.324 INFO:teuthology.orchestra.run.smithi191.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-07T12:57:50.325 INFO:teuthology.orchestra.run.smithi191.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-07T12:57:50.325 INFO:teuthology.orchestra.run.smithi191.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-07T12:57:50.325 INFO:teuthology.orchestra.run.smithi191.stdout:hardware devices. 2024-02-07T12:57:50.325 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-07T12:57:50.325 INFO:teuthology.orchestra.run.smithi191.stdout:This subpackage provides tools for support of all platforms. 2024-02-07T12:57:50.326 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-07T12:57:50.326 INFO:teuthology.task.kernel:Updating grub on smithi191 to boot 5.14.0-412.el9.x86_64 2024-02-07T12:57:50.326 DEBUG:teuthology.orchestra.run.smithi191:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-07T12:57:50.932 INFO:teuthology.orchestra.run.smithi191.stderr:Generating grub configuration file ... 2024-02-07T12:57:52.328 INFO:teuthology.orchestra.run.smithi191.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-07T12:57:52.357 INFO:teuthology.orchestra.run.smithi191.stderr:done 2024-02-07T12:57:52.361 DEBUG:teuthology.orchestra.run.smithi191:> mktemp 2024-02-07T12:57:52.375 INFO:teuthology.orchestra.run.smithi191.stdout:/tmp/tmp.lqKwF2NBPy 2024-02-07T12:57:52.376 DEBUG:teuthology.orchestra.run.smithi191:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.lqKwF2NBPy 2024-02-07T12:57:52.443 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0666 /tmp/tmp.lqKwF2NBPy 2024-02-07T12:57:52.584 DEBUG:teuthology.orchestra.remote:smithi191:/tmp/tmp.lqKwF2NBPy is 6KB 2024-02-07T12:57:52.633 DEBUG:teuthology.orchestra.run.smithi191:> rm -fr /tmp/tmp.lqKwF2NBPy 2024-02-07T12:57:52.647 DEBUG:teuthology.orchestra.run.smithi191:> sudo /bin/ls /boot/loader/entries || true 2024-02-07T12:57:52.719 INFO:teuthology.orchestra.run.smithi191.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-07T12:57:52.720 INFO:teuthology.orchestra.run.smithi191.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-07T12:57:52.720 INFO:teuthology.orchestra.run.smithi191.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-07T12:57:52.720 INFO:teuthology.orchestra.run.smithi191.stdout:15949e3c88704300882482eddfd7baae-5.14.0-412.el9.x86_64.conf 2024-02-07T12:57:52.721 DEBUG:teuthology.orchestra.run.smithi191:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-412.el9.x86_64 2024-02-07T12:57:52.872 DEBUG:teuthology.orchestra.run.smithi191:> sudo shutdown -r now 2024-02-07T12:58:02.279 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: kernel-core-5.14.0-412.el9.x86_64 4/4 2024-02-07T12:58:02.280 INFO:teuthology.orchestra.run.smithi159.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-07T12:58:02.280 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-02-07T12:58:02.364 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: kernel-modules-5.14.0-412.el9.x86_64 4/4 2024-02-07T12:58:02.853 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: kernel-5.14.0-412.el9.x86_64 4/4 2024-02-07T12:58:02.853 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : kernel-5.14.0-412.el9.x86_64 1/4 2024-02-07T12:58:02.853 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : kernel-core-5.14.0-412.el9.x86_64 2/4 2024-02-07T12:58:02.853 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : kernel-modules-5.14.0-412.el9.x86_64 3/4 2024-02-07T12:58:03.342 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : kernel-modules-core-5.14.0-412.el9.x86_64 4/4 2024-02-07T12:58:03.343 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-02-07T12:58:03.343 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-02-07T12:58:03.343 INFO:teuthology.orchestra.run.smithi159.stdout: kernel-5.14.0-412.el9.x86_64 2024-02-07T12:58:03.343 INFO:teuthology.orchestra.run.smithi159.stdout: kernel-core-5.14.0-412.el9.x86_64 2024-02-07T12:58:03.343 INFO:teuthology.orchestra.run.smithi159.stdout: kernel-modules-5.14.0-412.el9.x86_64 2024-02-07T12:58:03.343 INFO:teuthology.orchestra.run.smithi159.stdout: kernel-modules-core-5.14.0-412.el9.x86_64 2024-02-07T12:58:03.343 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-02-07T12:58:03.343 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-02-07T12:58:03.860 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q kernel | sort -rV | head -n 1 2024-02-07T12:58:04.016 INFO:teuthology.orchestra.run.smithi159.stdout:kernel-5.14.0-412.el9.x86_64 2024-02-07T12:58:04.016 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-412.el9.x86_64 2024-02-07T12:58:04.016 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-412.el9.x86_64 2024-02-07T12:58:04.017 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-07T12:58:04.017 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-07T12:58:04.017 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-07T12:58:04.017 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi159.front.sepia.ceph.com, path=None, version=distro) 2024-02-07T12:58:04.017 DEBUG:teuthology.orchestra.run.smithi159:> rpm -q kernel | sort -rV | head -n 1 2024-02-07T12:58:04.138 INFO:teuthology.orchestra.run.smithi159.stdout:kernel-5.14.0-412.el9.x86_64 2024-02-07T12:58:04.138 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-412.el9.x86_64 2024-02-07T12:58:04.138 DEBUG:teuthology.orchestra.run.smithi159:> sudo rpm -qi grub2-tools 2024-02-07T12:58:04.180 INFO:teuthology.orchestra.run.smithi159.stdout:Name : grub2-tools 2024-02-07T12:58:04.181 INFO:teuthology.orchestra.run.smithi159.stdout:Epoch : 1 2024-02-07T12:58:04.181 INFO:teuthology.orchestra.run.smithi159.stdout:Version : 2.06 2024-02-07T12:58:04.181 INFO:teuthology.orchestra.run.smithi159.stdout:Release : 68.el9 2024-02-07T12:58:04.181 INFO:teuthology.orchestra.run.smithi159.stdout:Architecture: x86_64 2024-02-07T12:58:04.181 INFO:teuthology.orchestra.run.smithi159.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-07T12:58:04.181 INFO:teuthology.orchestra.run.smithi159.stdout:Group : Unspecified 2024-02-07T12:58:04.181 INFO:teuthology.orchestra.run.smithi159.stdout:Size : 8264254 2024-02-07T12:58:04.181 INFO:teuthology.orchestra.run.smithi159.stdout:License : GPLv3+ 2024-02-07T12:58:04.181 INFO:teuthology.orchestra.run.smithi159.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-07T12:58:04.181 INFO:teuthology.orchestra.run.smithi159.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-07T12:58:04.181 INFO:teuthology.orchestra.run.smithi159.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-07T12:58:04.181 INFO:teuthology.orchestra.run.smithi159.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-07T12:58:04.181 INFO:teuthology.orchestra.run.smithi159.stdout:Packager : builder@centos.org 2024-02-07T12:58:04.182 INFO:teuthology.orchestra.run.smithi159.stdout:Vendor : CentOS 2024-02-07T12:58:04.182 INFO:teuthology.orchestra.run.smithi159.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-07T12:58:04.182 INFO:teuthology.orchestra.run.smithi159.stdout:Summary : Support tools for GRUB. 2024-02-07T12:58:04.182 INFO:teuthology.orchestra.run.smithi159.stdout:Description : 2024-02-07T12:58:04.182 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-02-07T12:58:04.182 INFO:teuthology.orchestra.run.smithi159.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-07T12:58:04.182 INFO:teuthology.orchestra.run.smithi159.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-07T12:58:04.182 INFO:teuthology.orchestra.run.smithi159.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-07T12:58:04.182 INFO:teuthology.orchestra.run.smithi159.stdout:hardware devices. 2024-02-07T12:58:04.182 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-02-07T12:58:04.182 INFO:teuthology.orchestra.run.smithi159.stdout:This subpackage provides tools for support of all platforms. 2024-02-07T12:58:04.183 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-07T12:58:04.183 INFO:teuthology.task.kernel:Updating grub on smithi159 to boot 5.14.0-412.el9.x86_64 2024-02-07T12:58:04.183 DEBUG:teuthology.orchestra.run.smithi159:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-07T12:58:04.834 INFO:teuthology.orchestra.run.smithi159.stderr:Generating grub configuration file ... 2024-02-07T12:58:06.170 INFO:teuthology.orchestra.run.smithi159.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-07T12:58:06.207 INFO:teuthology.orchestra.run.smithi159.stderr:done 2024-02-07T12:58:06.209 DEBUG:teuthology.orchestra.run.smithi159:> mktemp 2024-02-07T12:58:06.224 INFO:teuthology.orchestra.run.smithi159.stdout:/tmp/tmp.q1va9GTzhb 2024-02-07T12:58:06.224 DEBUG:teuthology.orchestra.run.smithi159:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.q1va9GTzhb 2024-02-07T12:58:06.292 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 0666 /tmp/tmp.q1va9GTzhb 2024-02-07T12:58:06.446 DEBUG:teuthology.orchestra.remote:smithi159:/tmp/tmp.q1va9GTzhb is 6KB 2024-02-07T12:58:06.494 DEBUG:teuthology.orchestra.run.smithi159:> rm -fr /tmp/tmp.q1va9GTzhb 2024-02-07T12:58:06.509 DEBUG:teuthology.orchestra.run.smithi159:> sudo /bin/ls /boot/loader/entries || true 2024-02-07T12:58:06.573 INFO:teuthology.orchestra.run.smithi159.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-07T12:58:06.573 INFO:teuthology.orchestra.run.smithi159.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-07T12:58:06.573 INFO:teuthology.orchestra.run.smithi159.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-07T12:58:06.573 INFO:teuthology.orchestra.run.smithi159.stdout:15949e3c88704300882482eddfd7baae-5.14.0-412.el9.x86_64.conf 2024-02-07T12:58:06.575 DEBUG:teuthology.orchestra.run.smithi159:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-412.el9.x86_64 2024-02-07T12:58:06.736 DEBUG:teuthology.orchestra.run.smithi159:> sudo shutdown -r now 2024-02-07T12:58:08.659 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-core-5.14.0-412.el9.x86_64 4/4 2024-02-07T12:58:08.659 INFO:teuthology.orchestra.run.smithi067.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-07T12:58:08.660 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T12:58:08.747 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-5.14.0-412.el9.x86_64 4/4 2024-02-07T12:58:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-5.14.0-412.el9.x86_64 4/4 2024-02-07T12:58:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-5.14.0-412.el9.x86_64 1/4 2024-02-07T12:58:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-core-5.14.0-412.el9.x86_64 2/4 2024-02-07T12:58:09.220 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-modules-5.14.0-412.el9.x86_64 3/4 2024-02-07T12:58:09.627 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-modules-core-5.14.0-412.el9.x86_64 4/4 2024-02-07T12:58:09.627 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T12:58:09.627 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-02-07T12:58:09.627 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-5.14.0-412.el9.x86_64 2024-02-07T12:58:09.627 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-core-5.14.0-412.el9.x86_64 2024-02-07T12:58:09.627 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules-5.14.0-412.el9.x86_64 2024-02-07T12:58:09.627 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules-core-5.14.0-412.el9.x86_64 2024-02-07T12:58:09.627 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T12:58:09.627 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-02-07T12:58:10.147 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q kernel | sort -rV | head -n 1 2024-02-07T12:58:10.339 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-5.14.0-412.el9.x86_64 2024-02-07T12:58:10.339 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-412.el9.x86_64 2024-02-07T12:58:10.340 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-412.el9.x86_64 2024-02-07T12:58:10.340 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-07T12:58:10.340 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-07T12:58:10.340 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-07T12:58:10.340 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi067.front.sepia.ceph.com, path=None, version=distro) 2024-02-07T12:58:10.340 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q kernel | sort -rV | head -n 1 2024-02-07T12:58:10.467 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-5.14.0-412.el9.x86_64 2024-02-07T12:58:10.467 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-412.el9.x86_64 2024-02-07T12:58:10.468 DEBUG:teuthology.orchestra.run.smithi067:> sudo rpm -qi grub2-tools 2024-02-07T12:58:10.505 INFO:teuthology.orchestra.run.smithi067.stdout:Name : grub2-tools 2024-02-07T12:58:10.505 INFO:teuthology.orchestra.run.smithi067.stdout:Epoch : 1 2024-02-07T12:58:10.505 INFO:teuthology.orchestra.run.smithi067.stdout:Version : 2.06 2024-02-07T12:58:10.505 INFO:teuthology.orchestra.run.smithi067.stdout:Release : 68.el9 2024-02-07T12:58:10.505 INFO:teuthology.orchestra.run.smithi067.stdout:Architecture: x86_64 2024-02-07T12:58:10.505 INFO:teuthology.orchestra.run.smithi067.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-07T12:58:10.506 INFO:teuthology.orchestra.run.smithi067.stdout:Group : Unspecified 2024-02-07T12:58:10.506 INFO:teuthology.orchestra.run.smithi067.stdout:Size : 8264254 2024-02-07T12:58:10.506 INFO:teuthology.orchestra.run.smithi067.stdout:License : GPLv3+ 2024-02-07T12:58:10.506 INFO:teuthology.orchestra.run.smithi067.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-07T12:58:10.506 INFO:teuthology.orchestra.run.smithi067.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-07T12:58:10.506 INFO:teuthology.orchestra.run.smithi067.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-07T12:58:10.506 INFO:teuthology.orchestra.run.smithi067.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-07T12:58:10.506 INFO:teuthology.orchestra.run.smithi067.stdout:Packager : builder@centos.org 2024-02-07T12:58:10.506 INFO:teuthology.orchestra.run.smithi067.stdout:Vendor : CentOS 2024-02-07T12:58:10.506 INFO:teuthology.orchestra.run.smithi067.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-07T12:58:10.506 INFO:teuthology.orchestra.run.smithi067.stdout:Summary : Support tools for GRUB. 2024-02-07T12:58:10.506 INFO:teuthology.orchestra.run.smithi067.stdout:Description : 2024-02-07T12:58:10.506 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T12:58:10.506 INFO:teuthology.orchestra.run.smithi067.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-07T12:58:10.507 INFO:teuthology.orchestra.run.smithi067.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-07T12:58:10.507 INFO:teuthology.orchestra.run.smithi067.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-07T12:58:10.507 INFO:teuthology.orchestra.run.smithi067.stdout:hardware devices. 2024-02-07T12:58:10.507 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T12:58:10.507 INFO:teuthology.orchestra.run.smithi067.stdout:This subpackage provides tools for support of all platforms. 2024-02-07T12:58:10.508 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-07T12:58:10.508 INFO:teuthology.task.kernel:Updating grub on smithi067 to boot 5.14.0-412.el9.x86_64 2024-02-07T12:58:10.508 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-07T12:58:11.144 INFO:teuthology.orchestra.run.smithi067.stderr:Generating grub configuration file ... 2024-02-07T12:58:12.613 INFO:teuthology.orchestra.run.smithi067.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-07T12:58:12.648 INFO:teuthology.orchestra.run.smithi067.stderr:done 2024-02-07T12:58:12.652 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-02-07T12:58:12.667 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.737kuw6KDN 2024-02-07T12:58:12.668 DEBUG:teuthology.orchestra.run.smithi067:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.737kuw6KDN 2024-02-07T12:58:12.739 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.737kuw6KDN 2024-02-07T12:58:12.896 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.737kuw6KDN is 6KB 2024-02-07T12:58:12.944 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.737kuw6KDN 2024-02-07T12:58:12.958 DEBUG:teuthology.orchestra.run.smithi067:> sudo /bin/ls /boot/loader/entries || true 2024-02-07T12:58:13.026 INFO:teuthology.orchestra.run.smithi067.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-07T12:58:13.026 INFO:teuthology.orchestra.run.smithi067.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-07T12:58:13.026 INFO:teuthology.orchestra.run.smithi067.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-07T12:58:13.026 INFO:teuthology.orchestra.run.smithi067.stdout:15949e3c88704300882482eddfd7baae-5.14.0-412.el9.x86_64.conf 2024-02-07T12:58:13.028 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-412.el9.x86_64 2024-02-07T12:58:13.184 DEBUG:teuthology.orchestra.run.smithi067:> sudo shutdown -r now 2024-02-07T12:58:22.880 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-02-07T12:58:22.880 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-02-07T12:58:22.881 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:58:36.752 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-07T12:58:36.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-02-07T12:58:36.753 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:58:43.192 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-07T12:58:43.192 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-02-07T12:58:43.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:59:11.296 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-02-07T12:59:20.304 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-02-07T12:59:20.305 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:59:22.884 DEBUG:teuthology.orchestra.remote:timed out 2024-02-07T12:59:31.892 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-02-07T12:59:31.893 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:59:35.564 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-02-07T12:59:36.013 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-02-07T12:59:36.013 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-412.el9.x86_64"... 2024-02-07T12:59:36.014 DEBUG:teuthology.orchestra.run.smithi191:> uname -r 2024-02-07T12:59:36.028 INFO:teuthology.orchestra.run.smithi191.stdout:5.14.0-412.el9.x86_64 2024-02-07T12:59:36.028 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-412.el9.x86_64 vs 5.14.0-412.el9.x86_64 2024-02-07T12:59:36.029 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-07T12:59:36.029 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-07T12:59:37.029 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-02-07T12:59:37.030 DEBUG:teuthology.orchestra.run.smithi191:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-07T12:59:37.081 INFO:teuthology.orchestra.run.smithi191.stdout:ttyS1 2024-02-07T12:59:37.116 DEBUG:teuthology.parallel:result is None 2024-02-07T12:59:38.688 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.159 2024-02-07T12:59:43.196 DEBUG:teuthology.orchestra.remote:timed out 2024-02-07T12:59:50.689 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-02-07T12:59:50.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:59:51.057 DEBUG:teuthology.orchestra.run.smithi159:> true 2024-02-07T12:59:51.470 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-02-07T12:59:51.470 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-412.el9.x86_64"... 2024-02-07T12:59:51.470 DEBUG:teuthology.orchestra.run.smithi159:> uname -r 2024-02-07T12:59:51.485 INFO:teuthology.orchestra.run.smithi159.stdout:5.14.0-412.el9.x86_64 2024-02-07T12:59:51.485 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-412.el9.x86_64 vs 5.14.0-412.el9.x86_64 2024-02-07T12:59:51.485 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-07T12:59:51.485 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-07T12:59:52.197 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-02-07T12:59:52.198 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T12:59:52.486 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-07T12:59:52.486 DEBUG:teuthology.orchestra.run.smithi159:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-07T12:59:52.553 INFO:teuthology.orchestra.run.smithi159.stdout:ttyS1 2024-02-07T12:59:52.604 DEBUG:teuthology.parallel:result is None 2024-02-07T12:59:55.260 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-02-07T13:00:07.261 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-02-07T13:00:07.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:00:07.884 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-02-07T13:00:08.513 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-02-07T13:00:08.513 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-412.el9.x86_64"... 2024-02-07T13:00:08.513 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-02-07T13:00:08.527 INFO:teuthology.orchestra.run.smithi067.stdout:5.14.0-412.el9.x86_64 2024-02-07T13:00:08.527 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-412.el9.x86_64 vs 5.14.0-412.el9.x86_64 2024-02-07T13:00:08.528 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-07T13:00:08.528 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-07T13:00:09.528 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-07T13:00:09.529 DEBUG:teuthology.orchestra.run.smithi067:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-07T13:00:09.608 INFO:teuthology.orchestra.run.smithi067.stdout:ttyS1 2024-02-07T13:00:09.648 DEBUG:teuthology.parallel:result is None 2024-02-07T13:00:09.649 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-07T13:00:09.661 INFO:teuthology.task.internal:Creating test directory... 2024-02-07T13:00:09.661 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-07T13:00:09.664 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-07T13:00:09.668 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-07T13:00:09.688 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-07T13:00:09.739 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-07T13:00:09.746 INFO:teuthology.task.internal:Creating archive directory... 2024-02-07T13:00:09.746 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-07T13:00:09.749 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-07T13:00:09.752 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-07T13:00:09.791 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-07T13:00:09.798 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-07T13:00:09.799 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-07T13:00:09.827 DEBUG:teuthology.orchestra.run.smithi159:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-07T13:00:09.830 DEBUG:teuthology.orchestra.run.smithi191:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-07T13:00:09.860 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-07T13:00:09.861 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-07T13:00:09.873 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-07T13:00:09.875 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-07T13:00:09.876 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-07T13:00:09.887 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-07T13:00:09.888 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-07T13:00:09.896 INFO:teuthology.task.internal:Configuring sudo... 2024-02-07T13:00:09.896 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-07T13:00:09.917 DEBUG:teuthology.orchestra.run.smithi159:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-07T13:00:09.920 DEBUG:teuthology.orchestra.run.smithi191:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-07T13:00:09.959 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-07T13:00:09.971 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-07T13:00:09.971 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-07T13:00:09.988 DEBUG:teuthology.orchestra.run.smithi159:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-07T13:00:09.991 DEBUG:teuthology.orchestra.run.smithi191:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-07T13:00:10.016 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-07T13:00:10.086 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-07T13:00:10.148 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-07T13:00:10.227 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-07T13:00:10.274 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-07T13:00:10.274 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-07T13:00:10.341 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-07T13:00:10.381 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-07T13:00:10.439 DEBUG:teuthology.orchestra.run.smithi159:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-07T13:00:10.523 DEBUG:teuthology.orchestra.run.smithi159:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-07T13:00:10.567 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-02-07T13:00:10.567 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-07T13:00:10.636 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-07T13:00:10.683 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-07T13:00:10.741 DEBUG:teuthology.orchestra.run.smithi191:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-07T13:00:10.784 DEBUG:teuthology.orchestra.run.smithi191:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-07T13:00:10.831 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-07T13:00:10.831 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-07T13:00:10.900 DEBUG:teuthology.orchestra.run.smithi067:> sudo service rsyslog restart 2024-02-07T13:00:10.903 DEBUG:teuthology.orchestra.run.smithi159:> sudo service rsyslog restart 2024-02-07T13:00:10.906 DEBUG:teuthology.orchestra.run.smithi191:> sudo service rsyslog restart 2024-02-07T13:00:10.945 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-07T13:00:10.955 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-07T13:00:10.990 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-07T13:00:11.466 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-07T13:00:11.474 INFO:teuthology.task.internal:Starting timer... 2024-02-07T13:00:11.475 INFO:teuthology.run_tasks:Running task pcp... 2024-02-07T13:00:11.516 INFO:teuthology.run_tasks:Running task selinux... 2024-02-07T13:00:11.528 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-07T13:00:11.529 DEBUG:teuthology.orchestra.run.smithi067:> sudo service auditd rotate 2024-02-07T13:00:11.588 INFO:teuthology.orchestra.run.smithi067.stdout:Rotating logs: 2024-02-07T13:00:11.590 DEBUG:teuthology.orchestra.run.smithi159:> sudo service auditd rotate 2024-02-07T13:00:11.663 INFO:teuthology.orchestra.run.smithi159.stdout:Rotating logs: 2024-02-07T13:00:11.665 DEBUG:teuthology.orchestra.run.smithi191:> sudo service auditd rotate 2024-02-07T13:00:11.727 INFO:teuthology.orchestra.run.smithi191.stdout:Rotating logs: 2024-02-07T13:00:11.729 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-07T13:00:11.729 DEBUG:teuthology.orchestra.run.smithi067:> /usr/sbin/getenforce 2024-02-07T13:00:11.759 INFO:teuthology.orchestra.run.smithi067.stdout:Permissive 2024-02-07T13:00:11.759 DEBUG:teuthology.orchestra.run.smithi159:> /usr/sbin/getenforce 2024-02-07T13:00:11.785 INFO:teuthology.orchestra.run.smithi159.stdout:Permissive 2024-02-07T13:00:11.785 DEBUG:teuthology.orchestra.run.smithi191:> /usr/sbin/getenforce 2024-02-07T13:00:11.803 INFO:teuthology.orchestra.run.smithi191.stdout:Permissive 2024-02-07T13:00:11.803 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi067.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi159.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi191.front.sepia.ceph.com': 'permissive'} 2024-02-07T13:00:11.804 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-07T13:00:11.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:00:11.833 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-07T13:00:11.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:00:11.863 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-07T13:00:11.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:00:11.892 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-07T13:00:11.892 DEBUG:teuthology.orchestra.run.smithi067:> sudo /usr/sbin/setenforce permissive 2024-02-07T13:00:11.928 DEBUG:teuthology.orchestra.run.smithi159:> sudo /usr/sbin/setenforce permissive 2024-02-07T13:00:11.962 DEBUG:teuthology.orchestra.run.smithi191:> sudo /usr/sbin/setenforce permissive 2024-02-07T13:00:11.995 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-07T13:00:12.006 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-07T13:00:12.022 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-07T13:00:12.024 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi067.front.sepia.ceph.com,smithi159.front.sepia.ceph.com,smithi191.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-07T13:04:53.809 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi067.front.sepia.ceph.com'), Remote(name='ubuntu@smithi159.front.sepia.ceph.com'), Remote(name='ubuntu@smithi191.front.sepia.ceph.com')] 2024-02-07T13:04:53.810 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-02-07T13:04:53.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:04:53.903 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-02-07T13:04:53.978 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-02-07T13:04:53.978 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-02-07T13:04:53.979 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi159.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:04:54.073 DEBUG:teuthology.orchestra.run.smithi159:> true 2024-02-07T13:04:54.156 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi159.front.sepia.ceph.com' 2024-02-07T13:04:54.156 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-02-07T13:04:54.157 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi191.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-07T13:04:54.249 DEBUG:teuthology.orchestra.run.smithi191:> true 2024-02-07T13:04:54.327 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi191.front.sepia.ceph.com' 2024-02-07T13:04:54.327 INFO:teuthology.run_tasks:Running task clock... 2024-02-07T13:04:54.339 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-07T13:04:54.339 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-07T13:04:54.340 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-07T13:04:54.342 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-07T13:04:54.342 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-07T13:04:54.344 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-07T13:04:54.345 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-07T13:04:54.371 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-07T13:04:54.377 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-07T13:04:54.388 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-07T13:04:54.395 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-07T13:04:54.398 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-07T13:04:54.415 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-07T13:04:54.417 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: ntpd: command not found 2024-02-07T13:04:54.426 INFO:teuthology.orchestra.run.smithi159.stderr:sudo: ntpd: command not found 2024-02-07T13:04:54.430 INFO:teuthology.orchestra.run.smithi067.stdout:506 Cannot talk to daemon 2024-02-07T13:04:54.440 INFO:teuthology.orchestra.run.smithi159.stdout:506 Cannot talk to daemon 2024-02-07T13:04:54.444 INFO:teuthology.orchestra.run.smithi191.stderr:sudo: ntpd: command not found 2024-02-07T13:04:54.446 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-07T13:04:54.457 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-07T13:04:54.457 INFO:teuthology.orchestra.run.smithi191.stdout:506 Cannot talk to daemon 2024-02-07T13:04:54.463 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-07T13:04:54.473 INFO:teuthology.orchestra.run.smithi159.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-07T13:04:54.474 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-07T13:04:54.490 INFO:teuthology.orchestra.run.smithi191.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-07T13:04:54.517 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2024-02-07T13:04:54.521 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-07T13:04:54.521 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-02-07T13:04:54.521 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:04:54.521 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:04:54.521 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:04:54.521 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:04:54.523 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-02-07T13:04:54.527 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-07T13:04:54.528 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-02-07T13:04:54.528 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:04:54.528 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:04:54.528 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:04:54.528 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:04:54.541 INFO:teuthology.orchestra.run.smithi191.stderr:bash: line 1: ntpq: command not found 2024-02-07T13:04:54.544 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-07T13:04:54.544 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2024-02-07T13:04:54.544 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:04:54.544 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:04:54.544 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:04:54.545 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:04:54.545 INFO:teuthology.run_tasks:Running task pexec... 2024-02-07T13:04:54.557 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-07T13:04:54.557 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi067.front.sepia.ceph.com 2024-02-07T13:04:54.557 DEBUG:teuthology.orchestra.run.smithi067:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-07T13:04:54.558 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi159.front.sepia.ceph.com 2024-02-07T13:04:54.558 DEBUG:teuthology.orchestra.run.smithi159:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-07T13:04:54.558 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi191.front.sepia.ceph.com 2024-02-07T13:04:54.558 DEBUG:teuthology.orchestra.run.smithi191:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-07T13:04:55.182 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:13 ago on Wed 07 Feb 2024 01:03:41 PM UTC. 2024-02-07T13:04:55.188 INFO:teuthology.orchestra.run.smithi159.stdout:Last metadata expiration check: 0:01:08 ago on Wed 07 Feb 2024 01:03:46 PM UTC. 2024-02-07T13:04:55.201 INFO:teuthology.orchestra.run.smithi191.stdout:Last metadata expiration check: 0:01:17 ago on Wed 07 Feb 2024 01:03:38 PM UTC. 2024-02-07T13:04:55.481 INFO:teuthology.orchestra.run.smithi159.stderr:Modular dependency problems: 2024-02-07T13:04:55.482 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-02-07T13:04:55.482 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 1: conflicting requests 2024-02-07T13:04:55.482 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240206092359:20125149.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.482 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 2: conflicting requests 2024-02-07T13:04:55.482 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.482 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 3: conflicting requests 2024-02-07T13:04:55.482 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.482 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 4: conflicting requests 2024-02-07T13:04:55.482 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.482 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 5: conflicting requests 2024-02-07T13:04:55.482 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820231031173453:b754926a.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.482 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 6: conflicting requests 2024-02-07T13:04:55.483 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820230817140043:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.483 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 7: conflicting requests 2024-02-07T13:04:55.483 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.483 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 8: conflicting requests 2024-02-07T13:04:55.483 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.483 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 9: conflicting requests 2024-02-07T13:04:55.483 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.483 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 10: conflicting requests 2024-02-07T13:04:55.483 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.483 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 11: conflicting requests 2024-02-07T13:04:55.483 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.483 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 12: conflicting requests 2024-02-07T13:04:55.483 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.483 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 13: conflicting requests 2024-02-07T13:04:55.484 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.484 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 14: conflicting requests 2024-02-07T13:04:55.484 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.484 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 15: conflicting requests 2024-02-07T13:04:55.484 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.484 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 16: conflicting requests 2024-02-07T13:04:55.484 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.484 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 17: conflicting requests 2024-02-07T13:04:55.484 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.484 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 18: conflicting requests 2024-02-07T13:04:55.484 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.484 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 19: conflicting requests 2024-02-07T13:04:55.484 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.484 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 20: conflicting requests 2024-02-07T13:04:55.485 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.485 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 21: conflicting requests 2024-02-07T13:04:55.485 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.485 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 22: conflicting requests 2024-02-07T13:04:55.485 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.485 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 23: conflicting requests 2024-02-07T13:04:55.485 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240202092102:182f7c73.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.485 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 24: conflicting requests 2024-02-07T13:04:55.485 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.485 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 25: conflicting requests 2024-02-07T13:04:55.485 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.485 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 26: conflicting requests 2024-02-07T13:04:55.485 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240117165712:17377f89.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.485 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 27: conflicting requests 2024-02-07T13:04:55.485 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.486 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 28: conflicting requests 2024-02-07T13:04:55.486 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.486 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 29: conflicting requests 2024-02-07T13:04:55.486 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.486 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 30: conflicting requests 2024-02-07T13:04:55.486 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.486 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 31: conflicting requests 2024-02-07T13:04:55.486 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.486 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 32: conflicting requests 2024-02-07T13:04:55.486 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module squid:4:820230817231530:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.486 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 33: conflicting requests 2024-02-07T13:04:55.486 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.486 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 34: conflicting requests 2024-02-07T13:04:55.486 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.486 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 35: conflicting requests 2024-02-07T13:04:55.486 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.487 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 36: conflicting requests 2024-02-07T13:04:55.487 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.487 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 37: conflicting requests 2024-02-07T13:04:55.487 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.487 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.487 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.487 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 38: cannot install the best candidate for the job 2024-02-07T13:04:55.487 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.487 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.487 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.487 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.487 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 39: conflicting requests 2024-02-07T13:04:55.487 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.487 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.487 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.488 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.488 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.488 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.488 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 40: cannot install the best candidate for the job 2024-02-07T13:04:55.488 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.488 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.488 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.488 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.488 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 41: cannot install the best candidate for the job 2024-02-07T13:04:55.488 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.488 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.488 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.488 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.489 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 42: cannot install the best candidate for the job 2024-02-07T13:04:55.489 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.489 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.489 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.489 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.489 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 43: cannot install the best candidate for the job 2024-02-07T13:04:55.489 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.489 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.489 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.491 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.491 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 44: cannot install the best candidate for the job 2024-02-07T13:04:55.491 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.491 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.491 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.492 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.492 INFO:teuthology.orchestra.run.smithi159.stderr: Problem 45: cannot install the best candidate for the job 2024-02-07T13:04:55.492 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.492 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.492 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.492 INFO:teuthology.orchestra.run.smithi159.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.492 INFO:teuthology.orchestra.run.smithi067.stderr:Modular dependency problems: 2024-02-07T13:04:55.492 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-07T13:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 1: conflicting requests 2024-02-07T13:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240206092359:20125149.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 2: conflicting requests 2024-02-07T13:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 3: conflicting requests 2024-02-07T13:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 4: conflicting requests 2024-02-07T13:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 5: conflicting requests 2024-02-07T13:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820231031173453:b754926a.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 6: conflicting requests 2024-02-07T13:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820230817140043:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 7: conflicting requests 2024-02-07T13:04:55.493 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.494 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 8: conflicting requests 2024-02-07T13:04:55.494 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.494 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 9: conflicting requests 2024-02-07T13:04:55.494 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.494 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 10: conflicting requests 2024-02-07T13:04:55.494 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.494 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 11: conflicting requests 2024-02-07T13:04:55.494 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.494 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 12: conflicting requests 2024-02-07T13:04:55.494 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.494 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 13: conflicting requests 2024-02-07T13:04:55.494 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.494 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 14: conflicting requests 2024-02-07T13:04:55.494 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.495 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 15: conflicting requests 2024-02-07T13:04:55.495 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.495 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 16: conflicting requests 2024-02-07T13:04:55.495 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.495 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 17: conflicting requests 2024-02-07T13:04:55.495 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.495 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 18: conflicting requests 2024-02-07T13:04:55.495 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.495 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 19: conflicting requests 2024-02-07T13:04:55.495 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.495 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 20: conflicting requests 2024-02-07T13:04:55.495 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.495 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 21: conflicting requests 2024-02-07T13:04:55.495 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.495 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 22: conflicting requests 2024-02-07T13:04:55.495 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 23: conflicting requests 2024-02-07T13:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240202092102:182f7c73.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 24: conflicting requests 2024-02-07T13:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 25: conflicting requests 2024-02-07T13:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 26: conflicting requests 2024-02-07T13:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240117165712:17377f89.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 27: conflicting requests 2024-02-07T13:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 28: conflicting requests 2024-02-07T13:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 29: conflicting requests 2024-02-07T13:04:55.496 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.497 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 30: conflicting requests 2024-02-07T13:04:55.497 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.497 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 31: conflicting requests 2024-02-07T13:04:55.497 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.497 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 32: conflicting requests 2024-02-07T13:04:55.497 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module squid:4:820230817231530:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.497 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 33: conflicting requests 2024-02-07T13:04:55.497 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.497 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 34: conflicting requests 2024-02-07T13:04:55.497 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.497 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 35: conflicting requests 2024-02-07T13:04:55.497 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.497 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 36: conflicting requests 2024-02-07T13:04:55.497 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.497 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 37: conflicting requests 2024-02-07T13:04:55.498 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.498 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.498 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.498 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 38: cannot install the best candidate for the job 2024-02-07T13:04:55.498 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.498 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.498 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.498 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.498 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 39: conflicting requests 2024-02-07T13:04:55.498 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.498 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.498 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.498 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.498 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.498 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.499 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 40: cannot install the best candidate for the job 2024-02-07T13:04:55.499 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.499 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.499 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.499 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.499 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 41: cannot install the best candidate for the job 2024-02-07T13:04:55.499 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.499 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.499 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.499 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.499 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 42: cannot install the best candidate for the job 2024-02-07T13:04:55.499 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.499 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.499 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.499 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.500 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 43: cannot install the best candidate for the job 2024-02-07T13:04:55.500 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.500 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.500 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.500 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.500 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 44: cannot install the best candidate for the job 2024-02-07T13:04:55.500 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.500 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.500 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.500 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.500 INFO:teuthology.orchestra.run.smithi067.stderr: Problem 45: cannot install the best candidate for the job 2024-02-07T13:04:55.500 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.500 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.500 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.500 INFO:teuthology.orchestra.run.smithi067.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.501 INFO:teuthology.orchestra.run.smithi191.stderr:Modular dependency problems: 2024-02-07T13:04:55.501 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-02-07T13:04:55.501 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 1: conflicting requests 2024-02-07T13:04:55.501 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module container-tools:rhel8:820240206092359:20125149.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.501 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 2: conflicting requests 2024-02-07T13:04:55.501 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module eclipse:rhel8:820201023100746:b230ed4e.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.501 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 3: conflicting requests 2024-02-07T13:04:55.501 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module freeradius:3.0:8080020230117180605:89170a74.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.502 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 4: conflicting requests 2024-02-07T13:04:55.502 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module gimp:2.8:8000020190628145146:4148dfdf.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.502 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 5: conflicting requests 2024-02-07T13:04:55.502 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module go-toolset:rhel8:820231031173453:b754926a.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.502 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 6: conflicting requests 2024-02-07T13:04:55.502 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module httpd:2.4:820230817140043:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.502 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 7: conflicting requests 2024-02-07T13:04:55.502 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module idm:client:820231122122737:49cc9d1b.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.502 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 8: conflicting requests 2024-02-07T13:04:55.502 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module inkscape:0.92.3:8080020221205124429:3e031279.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.502 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 9: conflicting requests 2024-02-07T13:04:55.502 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module jmc:rhel8:8050020211110222101:6392b1f8.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.502 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 10: conflicting requests 2024-02-07T13:04:55.502 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module llvm-toolset:rhel8:820240205133135:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.503 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 11: conflicting requests 2024-02-07T13:04:55.503 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module log4j:2:8080020221205124743:9d367344.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.503 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 12: conflicting requests 2024-02-07T13:04:55.503 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module mailman:2.1:820230727085757:77fc8825.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.503 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 13: conflicting requests 2024-02-07T13:04:55.503 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module mariadb:10.3:8030020210419150013:30b713e6.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.503 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 14: conflicting requests 2024-02-07T13:04:55.503 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module maven:3.5:8060020211203142015:c0229ad2.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.503 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 15: conflicting requests 2024-02-07T13:04:55.503 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module mercurial:4.8:8000020190628020724:4148dfdf.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.503 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 16: conflicting requests 2024-02-07T13:04:55.503 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module mod_auth_openidc:2.3:820230721123832:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.503 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 17: conflicting requests 2024-02-07T13:04:55.503 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module mysql:8.0:8040020210901180257:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.503 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 18: conflicting requests 2024-02-07T13:04:55.504 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module nginx:1.14:8000020211221191913:55190bc5.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.504 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 19: conflicting requests 2024-02-07T13:04:55.504 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module nodejs:10:8040020210120182536:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.504 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 20: conflicting requests 2024-02-07T13:04:55.504 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl:5.26:8000020190628020724:55190bc5.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.504 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 21: conflicting requests 2024-02-07T13:04:55.504 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module php:7.2:8020020200507003613:2c7ca891.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.504 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 22: conflicting requests 2024-02-07T13:04:55.504 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module postgresql:10:820230223114800:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.504 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 23: conflicting requests 2024-02-07T13:04:55.504 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module python27:2.7:820240202092102:182f7c73.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.504 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 24: conflicting requests 2024-02-07T13:04:55.504 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module python36:3.6:8050020210825152031:982725ab.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.504 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 25: conflicting requests 2024-02-07T13:04:55.505 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module python38:3.8:820230810143931:64c1cd5a.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.505 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 26: conflicting requests 2024-02-07T13:04:55.505 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module python39:3.9:820240117165712:17377f89.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.505 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 27: conflicting requests 2024-02-07T13:04:55.505 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module redis:5:8040020211019153849:9f9e2e7e.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.505 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 28: conflicting requests 2024-02-07T13:04:55.505 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module rhn-tools:1.0:8010020191114034948:f69d1239.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.505 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 29: conflicting requests 2024-02-07T13:04:55.505 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module ruby:2.5:820230627084142:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.505 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 30: conflicting requests 2024-02-07T13:04:55.505 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module rust-toolset:rhel8:820240119204620:b09eea91.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.505 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 31: conflicting requests 2024-02-07T13:04:55.505 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module satellite-5-client:1.0:8010020191114035551:cdc1202b.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.505 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 32: conflicting requests 2024-02-07T13:04:55.505 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module squid:4:820230817231530:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.506 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 33: conflicting requests 2024-02-07T13:04:55.506 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module subversion:1.10:8070020220711155714:78111232.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.506 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 34: conflicting requests 2024-02-07T13:04:55.506 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module swig:3.0:8030020200706142531:30b713e6.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.506 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 35: conflicting requests 2024-02-07T13:04:55.506 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module varnish:6:8050020211109225449:b4937e53.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.506 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 36: conflicting requests 2024-02-07T13:04:55.506 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module virt:rhel:820240203221207:9edba152.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.506 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 37: conflicting requests 2024-02-07T13:04:55.506 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:7cc0a66d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.506 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:8030020200716155257:b967a9a2.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.506 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-libwww-perl:6.34:820230811133638:36f9fae6.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.506 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 38: cannot install the best candidate for the job 2024-02-07T13:04:55.506 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:a5949e2e.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.506 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:8010020191114031501:be2e4737.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.506 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:2ab761e1.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.507 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-YAML:1.24:820230223114759:86e45846.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.507 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 39: conflicting requests 2024-02-07T13:04:55.507 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.507 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:8030020200715230104:ea09926d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.507 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:082fdf2f.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.507 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.507 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.507 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-IO-Socket-SSL:2.066:820230223114759:fbe42456.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.507 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 40: cannot install the best candidate for the job 2024-02-07T13:04:55.507 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:16b3ab4d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.507 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:8010020191114031513:e1f37755.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.507 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.507 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-FCGI:0.78:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.507 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 41: cannot install the best candidate for the job 2024-02-07T13:04:55.508 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:1e4bbb35.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.508 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:8030020200716150652:ea09926d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.508 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:0ccef39c.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.508 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBI:1.641:820230223114759:dbc5bc9a.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.508 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 42: cannot install the best candidate for the job 2024-02-07T13:04:55.508 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:3a70019f.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.508 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:8030020200716174729:cccafca5.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.508 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.508 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-SQLite:1.58:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.508 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 43: cannot install the best candidate for the job 2024-02-07T13:04:55.508 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:56fce90f.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.508 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:8030020200716213634:98a3c9d0.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.508 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:06b7596e.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.508 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-Pg:3.7:820230223114759:d2fc5d76.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.509 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 44: cannot install the best candidate for the job 2024-02-07T13:04:55.509 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:3a70019f.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.509 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:8030020200716171139:cccafca5.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.509 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:6fdf7e5d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.509 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-DBD-MySQL:4.046:820230223114759:d4326aba.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.509 INFO:teuthology.orchestra.run.smithi191.stderr: Problem 45: cannot install the best candidate for the job 2024-02-07T13:04:55.509 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8010020191120175858:95fce66d.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.509 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:8030020200723233049:09acf126.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.509 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:436af1ae.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.509 INFO:teuthology.orchestra.run.smithi191.stderr: - nothing provides module(platform:el8) needed by module perl-App-cpanminus:1.7044:820230223114759:b12195ea.x86_64 from CentOS-AppStream 2024-02-07T13:04:55.548 INFO:teuthology.orchestra.run.smithi159.stdout:Dependencies resolved. 2024-02-07T13:04:55.558 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-02-07T13:04:55.560 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-02-07T13:04:55.560 INFO:teuthology.orchestra.run.smithi067.stdout: Package Architecture Version Repository Size 2024-02-07T13:04:55.561 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-02-07T13:04:55.561 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-02-07T13:04:55.561 INFO:teuthology.orchestra.run.smithi067.stdout: runc x86_64 4:1.1.11-1.el9 appstream 3.1 M 2024-02-07T13:04:55.561 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:04:55.561 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-02-07T13:04:55.561 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-02-07T13:04:55.561 INFO:teuthology.orchestra.run.smithi067.stdout:Install 1 Package 2024-02-07T13:04:55.561 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:04:55.562 INFO:teuthology.orchestra.run.smithi191.stdout:Dependencies resolved. 2024-02-07T13:04:55.562 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 3.1 M 2024-02-07T13:04:55.562 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 9.8 M 2024-02-07T13:04:55.562 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-02-07T13:04:55.563 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-02-07T13:04:55.563 INFO:teuthology.orchestra.run.smithi159.stdout: Package Architecture Version Repository Size 2024-02-07T13:04:55.563 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-02-07T13:04:55.563 INFO:teuthology.orchestra.run.smithi159.stdout:Installing: 2024-02-07T13:04:55.563 INFO:teuthology.orchestra.run.smithi159.stdout: runc x86_64 4:1.1.11-1.el9 appstream 3.1 M 2024-02-07T13:04:55.563 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-02-07T13:04:55.563 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction Summary 2024-02-07T13:04:55.563 INFO:teuthology.orchestra.run.smithi159.stdout:================================================================================ 2024-02-07T13:04:55.563 INFO:teuthology.orchestra.run.smithi159.stdout:Install 1 Package 2024-02-07T13:04:55.563 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-02-07T13:04:55.563 INFO:teuthology.orchestra.run.smithi159.stdout:Total download size: 3.1 M 2024-02-07T13:04:55.563 INFO:teuthology.orchestra.run.smithi159.stdout:Installed size: 9.8 M 2024-02-07T13:04:55.563 INFO:teuthology.orchestra.run.smithi159.stdout:Downloading Packages: 2024-02-07T13:04:55.564 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-07T13:04:55.564 INFO:teuthology.orchestra.run.smithi191.stdout: Package Architecture Version Repository Size 2024-02-07T13:04:55.564 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-07T13:04:55.564 INFO:teuthology.orchestra.run.smithi191.stdout:Installing: 2024-02-07T13:04:55.564 INFO:teuthology.orchestra.run.smithi191.stdout: runc x86_64 4:1.1.11-1.el9 appstream 3.1 M 2024-02-07T13:04:55.564 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-07T13:04:55.565 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction Summary 2024-02-07T13:04:55.565 INFO:teuthology.orchestra.run.smithi191.stdout:================================================================================ 2024-02-07T13:04:55.565 INFO:teuthology.orchestra.run.smithi191.stdout:Install 1 Package 2024-02-07T13:04:55.565 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-07T13:04:55.565 INFO:teuthology.orchestra.run.smithi191.stdout:Total download size: 3.1 M 2024-02-07T13:04:55.565 INFO:teuthology.orchestra.run.smithi191.stdout:Installed size: 9.8 M 2024-02-07T13:04:55.565 INFO:teuthology.orchestra.run.smithi191.stdout:Downloading Packages: 2024-02-07T13:04:56.088 INFO:teuthology.orchestra.run.smithi067.stdout:runc-1.1.11-1.el9.x86_64.rpm 8.8 MB/s | 3.1 MB 00:00 2024-02-07T13:04:56.088 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-02-07T13:04:56.088 INFO:teuthology.orchestra.run.smithi067.stdout:Total 5.9 MB/s | 3.1 MB 00:00 2024-02-07T13:04:56.124 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-02-07T13:04:56.131 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-02-07T13:04:56.131 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-02-07T13:04:56.213 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-02-07T13:04:56.213 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-02-07T13:04:56.430 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-02-07T13:04:56.535 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : runc-4:1.1.11-1.el9.x86_64 1/1 2024-02-07T13:04:56.700 INFO:teuthology.orchestra.run.smithi191.stdout:runc-1.1.11-1.el9.x86_64.rpm 2.9 MB/s | 3.1 MB 00:01 2024-02-07T13:04:56.700 INFO:teuthology.orchestra.run.smithi191.stdout:-------------------------------------------------------------------------------- 2024-02-07T13:04:56.700 INFO:teuthology.orchestra.run.smithi191.stdout:Total 2.8 MB/s | 3.1 MB 00:01 2024-02-07T13:04:56.737 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction check 2024-02-07T13:04:56.743 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction check succeeded. 2024-02-07T13:04:56.743 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction test 2024-02-07T13:04:56.767 INFO:teuthology.orchestra.run.smithi159.stdout:runc-1.1.11-1.el9.x86_64.rpm 2.7 MB/s | 3.1 MB 00:01 2024-02-07T13:04:56.767 INFO:teuthology.orchestra.run.smithi159.stdout:-------------------------------------------------------------------------------- 2024-02-07T13:04:56.768 INFO:teuthology.orchestra.run.smithi159.stdout:Total 2.6 MB/s | 3.1 MB 00:01 2024-02-07T13:04:56.807 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction check 2024-02-07T13:04:56.813 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction check succeeded. 2024-02-07T13:04:56.813 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction test 2024-02-07T13:04:56.824 INFO:teuthology.orchestra.run.smithi191.stdout:Transaction test succeeded. 2024-02-07T13:04:56.825 INFO:teuthology.orchestra.run.smithi191.stdout:Running transaction 2024-02-07T13:04:56.894 INFO:teuthology.orchestra.run.smithi159.stdout:Transaction test succeeded. 2024-02-07T13:04:56.895 INFO:teuthology.orchestra.run.smithi159.stdout:Running transaction 2024-02-07T13:04:57.025 INFO:teuthology.orchestra.run.smithi191.stdout: Preparing : 1/1 2024-02-07T13:04:57.104 INFO:teuthology.orchestra.run.smithi159.stdout: Preparing : 1/1 2024-02-07T13:04:57.130 INFO:teuthology.orchestra.run.smithi191.stdout: Installing : runc-4:1.1.11-1.el9.x86_64 1/1 2024-02-07T13:04:57.139 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: runc-4:1.1.11-1.el9.x86_64 1/1 2024-02-07T13:04:57.187 INFO:teuthology.orchestra.run.smithi159.stdout: Installing : runc-4:1.1.11-1.el9.x86_64 1/1 2024-02-07T13:04:57.474 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : runc-4:1.1.11-1.el9.x86_64 1/1 2024-02-07T13:04:57.474 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:04:57.474 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-02-07T13:04:57.474 INFO:teuthology.orchestra.run.smithi067.stdout: runc-4:1.1.11-1.el9.x86_64 2024-02-07T13:04:57.474 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:04:57.475 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-02-07T13:04:57.712 INFO:teuthology.orchestra.run.smithi159.stdout: Running scriptlet: runc-4:1.1.11-1.el9.x86_64 1/1 2024-02-07T13:04:57.736 DEBUG:teuthology.parallel:result is None 2024-02-07T13:04:57.744 INFO:teuthology.orchestra.run.smithi191.stdout: Running scriptlet: runc-4:1.1.11-1.el9.x86_64 1/1 2024-02-07T13:04:58.069 INFO:teuthology.orchestra.run.smithi191.stdout: Verifying : runc-4:1.1.11-1.el9.x86_64 1/1 2024-02-07T13:04:58.069 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-07T13:04:58.069 INFO:teuthology.orchestra.run.smithi191.stdout:Installed: 2024-02-07T13:04:58.069 INFO:teuthology.orchestra.run.smithi191.stdout: runc-4:1.1.11-1.el9.x86_64 2024-02-07T13:04:58.069 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-07T13:04:58.069 INFO:teuthology.orchestra.run.smithi191.stdout:Complete! 2024-02-07T13:04:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: Verifying : runc-4:1.1.11-1.el9.x86_64 1/1 2024-02-07T13:04:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-02-07T13:04:58.070 INFO:teuthology.orchestra.run.smithi159.stdout:Installed: 2024-02-07T13:04:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: runc-4:1.1.11-1.el9.x86_64 2024-02-07T13:04:58.070 INFO:teuthology.orchestra.run.smithi159.stdout: 2024-02-07T13:04:58.070 INFO:teuthology.orchestra.run.smithi159.stdout:Complete! 2024-02-07T13:04:58.340 DEBUG:teuthology.parallel:result is None 2024-02-07T13:04:58.345 DEBUG:teuthology.parallel:result is None 2024-02-07T13:04:58.345 INFO:teuthology.run_tasks:Running task nvme_loop... 2024-02-07T13:04:58.357 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2024-02-07T13:04:58.357 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-07T13:04:58.357 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T13:04:58.374 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-07T13:04:58.374 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_1 2024-02-07T13:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-07T13:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-02-07T13:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-02-07 13:04:52.714012426 +0000 2024-02-07T13:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-02-07 13:04:07.999036345 +0000 2024-02-07T13:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-02-07 13:04:07.999036345 +0000 2024-02-07T13:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-02-07 13:04:07.999036345 +0000 2024-02-07T13:04:58.433 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-07T13:04:58.501 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-02-07T13:04:58.501 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-02-07T13:04:58.501 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000259865 s, 2.0 MB/s 2024-02-07T13:04:58.502 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-07T13:04:58.560 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_2 2024-02-07T13:04:58.617 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-07T13:04:58.617 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:04:58.617 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-02-07T13:04:58.617 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:04:58.617 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:04:58.617 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-02-07 13:04:52.714012426 +0000 2024-02-07T13:04:58.617 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-02-07 13:04:08.345028417 +0000 2024-02-07T13:04:58.617 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-02-07 13:04:08.345028417 +0000 2024-02-07T13:04:58.618 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-02-07 13:04:08.345028417 +0000 2024-02-07T13:04:58.618 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-07T13:04:58.684 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-02-07T13:04:58.684 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-02-07T13:04:58.684 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000249533 s, 2.1 MB/s 2024-02-07T13:04:58.685 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-07T13:04:58.742 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_3 2024-02-07T13:04:58.798 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-07T13:04:58.799 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:04:58.799 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-02-07T13:04:58.799 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:04:58.799 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:04:58.799 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-02-07 13:04:52.714012426 +0000 2024-02-07T13:04:58.799 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-02-07 13:04:08.684020650 +0000 2024-02-07T13:04:58.799 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-02-07 13:04:08.684020650 +0000 2024-02-07T13:04:58.799 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-02-07 13:04:08.684020650 +0000 2024-02-07T13:04:58.799 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-07T13:04:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-02-07T13:04:58.865 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-02-07T13:04:58.866 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000185875 s, 2.8 MB/s 2024-02-07T13:04:58.866 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-07T13:04:58.922 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_4 2024-02-07T13:04:58.977 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-07T13:04:58.978 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:04:58.978 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-02-07T13:04:58.978 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:04:58.978 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:04:58.978 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-02-07 13:04:52.714012426 +0000 2024-02-07T13:04:58.978 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-02-07 13:04:09.020012951 +0000 2024-02-07T13:04:58.978 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-02-07 13:04:09.020012951 +0000 2024-02-07T13:04:58.978 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-02-07 13:04:09.020012951 +0000 2024-02-07T13:04:58.978 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-07T13:04:59.044 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-02-07T13:04:59.044 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-02-07T13:04:59.044 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000182648 s, 2.8 MB/s 2024-02-07T13:04:59.045 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-07T13:04:59.100 DEBUG:teuthology.orchestra.run.smithi067:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-07T13:04:59.244 INFO:teuthology.orchestra.run.smithi067.stdout:loop 2024-02-07T13:04:59.245 INFO:tasks.nvme_loop:Connecting nvme_loop smithi067:/dev/vg_nvme/lv_1... 2024-02-07T13:04:59.245 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-07T13:04:59.282 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-02-07T13:04:59.319 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/vg_nvme/lv_11 2024-02-07T13:04:59.356 INFO:tasks.nvme_loop:Connecting nvme_loop smithi067:/dev/vg_nvme/lv_2... 2024-02-07T13:04:59.356 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-07T13:04:59.438 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-02-07T13:04:59.476 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/vg_nvme/lv_21 2024-02-07T13:04:59.504 INFO:tasks.nvme_loop:Connecting nvme_loop smithi067:/dev/vg_nvme/lv_3... 2024-02-07T13:04:59.504 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-07T13:04:59.580 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-02-07T13:04:59.618 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/vg_nvme/lv_31 2024-02-07T13:04:59.645 INFO:tasks.nvme_loop:Connecting nvme_loop smithi067:/dev/vg_nvme/lv_4... 2024-02-07T13:04:59.645 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-07T13:04:59.722 INFO:teuthology.orchestra.run.smithi067.stdout:1 2024-02-07T13:04:59.760 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/vg_nvme/lv_41 2024-02-07T13:04:59.788 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-07T13:04:59.788 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T13:04:59.842 DEBUG:teuthology.orchestra.run.smithi067:> sudo nvme list 2024-02-07T13:04:59.908 INFO:teuthology.orchestra.run.smithi067.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-07T13:04:59.908 INFO:teuthology.orchestra.run.smithi067.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-07T13:04:59.908 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/nvme0n1 CVFT53440042400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2024-02-07T13:04:59.908 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/nvme1n1 1ed150da14ac69614e34 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:04:59.908 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/nvme2n1 b4027e7d11be2526586f Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:04:59.908 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/nvme3n1 f627b8f5491ef3b8ef19 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:04:59.908 INFO:teuthology.orchestra.run.smithi067.stdout:/dev/nvme4n1 4f2c04494dc72fca7467 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:04:59.909 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-07T13:04:59.909 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-07T13:04:59.909 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/scratch_devs 2024-02-07T13:04:59.976 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-02-07T13:04:59.976 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T13:04:59.995 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-07T13:04:59.995 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_1 2024-02-07T13:05:00.053 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-07T13:05:00.054 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:05:00.054 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-02-07T13:05:00.054 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:05:00.054 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:05:00.054 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-02-07 13:04:52.714574981 +0000 2024-02-07T13:05:00.054 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-02-07 13:04:13.670315342 +0000 2024-02-07T13:05:00.054 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-02-07 13:04:13.670315342 +0000 2024-02-07T13:05:00.054 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-02-07 13:04:13.670315342 +0000 2024-02-07T13:05:00.054 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-07T13:05:00.122 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-02-07T13:05:00.123 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-02-07T13:05:00.123 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000225839 s, 2.3 MB/s 2024-02-07T13:05:00.124 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-07T13:05:00.181 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_2 2024-02-07T13:05:00.237 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-07T13:05:00.237 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:05:00.237 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-02-07T13:05:00.237 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:05:00.237 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:05:00.238 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-02-07 13:04:52.714574981 +0000 2024-02-07T13:05:00.238 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-02-07 13:04:14.049308132 +0000 2024-02-07T13:05:00.238 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-02-07 13:04:14.049308132 +0000 2024-02-07T13:05:00.238 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-02-07 13:04:14.049308132 +0000 2024-02-07T13:05:00.238 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-07T13:05:00.305 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-02-07T13:05:00.306 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-02-07T13:05:00.306 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000216252 s, 2.4 MB/s 2024-02-07T13:05:00.307 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-07T13:05:00.364 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_3 2024-02-07T13:05:00.419 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-07T13:05:00.420 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:05:00.420 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-02-07T13:05:00.420 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:05:00.420 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:05:00.420 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-02-07 13:04:52.714574981 +0000 2024-02-07T13:05:00.420 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-02-07 13:04:14.398301515 +0000 2024-02-07T13:05:00.420 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-02-07 13:04:14.398301515 +0000 2024-02-07T13:05:00.420 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-02-07 13:04:14.398301515 +0000 2024-02-07T13:05:00.420 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-07T13:05:00.486 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-02-07T13:05:00.486 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-02-07T13:05:00.486 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000216809 s, 2.4 MB/s 2024-02-07T13:05:00.487 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-07T13:05:00.544 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/vg_nvme/lv_4 2024-02-07T13:05:00.599 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-07T13:05:00.599 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:05:00.599 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-02-07T13:05:00.599 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:05:00.599 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:05:00.600 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-02-07 13:04:52.714574981 +0000 2024-02-07T13:05:00.600 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-02-07 13:04:14.743294973 +0000 2024-02-07T13:05:00.600 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-02-07 13:04:14.743294973 +0000 2024-02-07T13:05:00.600 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-02-07 13:04:14.743294973 +0000 2024-02-07T13:05:00.600 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-07T13:05:00.664 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-02-07T13:05:00.665 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-02-07T13:05:00.665 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000226539 s, 2.3 MB/s 2024-02-07T13:05:00.666 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-07T13:05:00.722 DEBUG:teuthology.orchestra.run.smithi159:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-07T13:05:00.865 INFO:teuthology.orchestra.run.smithi159.stdout:loop 2024-02-07T13:05:00.867 INFO:tasks.nvme_loop:Connecting nvme_loop smithi159:/dev/vg_nvme/lv_1... 2024-02-07T13:05:00.867 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-07T13:05:00.904 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-02-07T13:05:00.942 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/vg_nvme/lv_11 2024-02-07T13:05:00.971 INFO:tasks.nvme_loop:Connecting nvme_loop smithi159:/dev/vg_nvme/lv_2... 2024-02-07T13:05:00.971 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-07T13:05:01.054 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-02-07T13:05:01.094 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/vg_nvme/lv_21 2024-02-07T13:05:01.124 INFO:tasks.nvme_loop:Connecting nvme_loop smithi159:/dev/vg_nvme/lv_3... 2024-02-07T13:05:01.124 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-07T13:05:01.203 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-02-07T13:05:01.242 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/vg_nvme/lv_31 2024-02-07T13:05:01.271 INFO:tasks.nvme_loop:Connecting nvme_loop smithi159:/dev/vg_nvme/lv_4... 2024-02-07T13:05:01.271 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-07T13:05:01.350 INFO:teuthology.orchestra.run.smithi159.stdout:1 2024-02-07T13:05:01.389 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/vg_nvme/lv_41 2024-02-07T13:05:01.420 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-02-07T13:05:01.420 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T13:05:01.476 DEBUG:teuthology.orchestra.run.smithi159:> sudo nvme list 2024-02-07T13:05:01.543 INFO:teuthology.orchestra.run.smithi159.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-07T13:05:01.543 INFO:teuthology.orchestra.run.smithi159.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-07T13:05:01.543 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/nvme0n1 CVFT623300GH400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2024-02-07T13:05:01.543 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/nvme1n1 c46879dfccccf212f527 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:05:01.543 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/nvme2n1 457c5bb7b695c5a9cff9 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:05:01.544 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/nvme3n1 6d79fe0dd1759adaa4db Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:05:01.544 INFO:teuthology.orchestra.run.smithi159.stdout:/dev/nvme4n1 0906f27ba2f040807628 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:05:01.544 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-07T13:05:01.545 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-02-07T13:05:01.545 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/scratch_devs 2024-02-07T13:05:01.613 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-07T13:05:01.613 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T13:05:01.631 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-07T13:05:01.632 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_1 2024-02-07T13:05:01.689 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-07T13:05:01.689 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:05:01.689 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-02-07T13:05:01.689 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:05:01.690 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:05:01.690 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-07 13:04:52.795908299 +0000 2024-02-07T13:05:01.690 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-07 13:04:05.652699322 +0000 2024-02-07T13:05:01.690 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-07 13:04:05.652699322 +0000 2024-02-07T13:05:01.690 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-02-07 13:04:05.652699322 +0000 2024-02-07T13:05:01.690 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-07T13:05:01.760 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-07T13:05:01.760 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-07T13:05:01.760 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000195974 s, 2.6 MB/s 2024-02-07T13:05:01.761 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-07T13:05:01.818 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_2 2024-02-07T13:05:01.875 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-07T13:05:01.875 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:05:01.875 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-02-07T13:05:01.876 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:05:01.876 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:05:01.876 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-07 13:04:52.795908299 +0000 2024-02-07T13:05:01.876 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-07 13:04:06.007693365 +0000 2024-02-07T13:05:01.876 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-07 13:04:06.007693365 +0000 2024-02-07T13:05:01.876 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-02-07 13:04:06.007693365 +0000 2024-02-07T13:05:01.876 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-07T13:05:01.943 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-07T13:05:01.943 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-07T13:05:01.943 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000195371 s, 2.6 MB/s 2024-02-07T13:05:01.945 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-07T13:05:02.002 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_3 2024-02-07T13:05:02.058 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-07T13:05:02.058 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:05:02.058 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-02-07T13:05:02.058 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:05:02.058 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:05:02.058 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-07 13:04:52.795908299 +0000 2024-02-07T13:05:02.058 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-07 13:04:06.363687392 +0000 2024-02-07T13:05:02.058 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-07 13:04:06.363687392 +0000 2024-02-07T13:05:02.058 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-02-07 13:04:06.363687392 +0000 2024-02-07T13:05:02.058 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-07T13:05:02.124 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-07T13:05:02.124 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-07T13:05:02.124 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000196305 s, 2.6 MB/s 2024-02-07T13:05:02.126 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-07T13:05:02.182 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/vg_nvme/lv_4 2024-02-07T13:05:02.237 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-07T13:05:02.237 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-07T13:05:02.237 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-07T13:05:02.237 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-07T13:05:02.237 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:device_t:s0 2024-02-07T13:05:02.237 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-07 13:04:52.795908299 +0000 2024-02-07T13:05:02.237 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-07 13:04:06.741681049 +0000 2024-02-07T13:05:02.237 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-07 13:04:06.741681049 +0000 2024-02-07T13:05:02.237 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-02-07 13:04:06.741681049 +0000 2024-02-07T13:05:02.237 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-07T13:05:02.303 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-07T13:05:02.303 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-07T13:05:02.303 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000175088 s, 2.9 MB/s 2024-02-07T13:05:02.305 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-07T13:05:02.360 DEBUG:teuthology.orchestra.run.smithi191:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-07T13:05:02.508 INFO:teuthology.orchestra.run.smithi191.stdout:loop 2024-02-07T13:05:02.510 INFO:tasks.nvme_loop:Connecting nvme_loop smithi191:/dev/vg_nvme/lv_1... 2024-02-07T13:05:02.510 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-07T13:05:02.547 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-02-07T13:05:02.585 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/vg_nvme/lv_11 2024-02-07T13:05:02.622 INFO:tasks.nvme_loop:Connecting nvme_loop smithi191:/dev/vg_nvme/lv_2... 2024-02-07T13:05:02.622 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-07T13:05:02.701 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-02-07T13:05:02.740 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/vg_nvme/lv_21 2024-02-07T13:05:02.776 INFO:tasks.nvme_loop:Connecting nvme_loop smithi191:/dev/vg_nvme/lv_3... 2024-02-07T13:05:02.776 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-07T13:05:02.854 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-02-07T13:05:02.892 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/vg_nvme/lv_31 2024-02-07T13:05:02.920 INFO:tasks.nvme_loop:Connecting nvme_loop smithi191:/dev/vg_nvme/lv_4... 2024-02-07T13:05:02.920 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-07T13:05:02.999 INFO:teuthology.orchestra.run.smithi191.stdout:1 2024-02-07T13:05:03.037 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/vg_nvme/lv_41 2024-02-07T13:05:03.065 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-07T13:05:03.065 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T13:05:03.126 DEBUG:teuthology.orchestra.run.smithi191:> sudo nvme list 2024-02-07T13:05:03.191 INFO:teuthology.orchestra.run.smithi191.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-07T13:05:03.191 INFO:teuthology.orchestra.run.smithi191.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-07T13:05:03.191 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/nvme0n1 CVFT6303008M400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2024-02-07T13:05:03.191 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/nvme1n1 468ca342ac84082eabee Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:05:03.192 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/nvme2n1 aa3cdf8222426e25d7a2 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:05:03.192 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/nvme3n1 98bc860a0cce584729f6 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:05:03.192 INFO:teuthology.orchestra.run.smithi191.stdout:/dev/nvme4n1 8ed4b22ce8533619e02b Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-07T13:05:03.193 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-07T13:05:03.193 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-07T13:05:03.193 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/scratch_devs 2024-02-07T13:05:03.259 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-07T13:05:03.356 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': True}, 'global': {'mon election default strategy': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '44458ccedb18241cc2e17f10cdab3d1385399261'} 2024-02-07T13:05:03.357 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T13:05:03.357 INFO:tasks.cephadm:Cluster fsid is 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:05:03.357 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-07T13:05:03.358 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.67', 'mon.b': '172.21.15.159', 'mon.c': '172.21.15.191'} 2024-02-07T13:05:03.358 INFO:tasks.cephadm:First mon is mon.a on smithi067 2024-02-07T13:05:03.358 INFO:tasks.cephadm:First mgr is y 2024-02-07T13:05:03.358 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-07T13:05:03.358 DEBUG:teuthology.orchestra.run.smithi067:> sudo hostname $(hostname -s) 2024-02-07T13:05:03.390 DEBUG:teuthology.orchestra.run.smithi159:> sudo hostname $(hostname -s) 2024-02-07T13:05:03.422 DEBUG:teuthology.orchestra.run.smithi191:> sudo hostname $(hostname -s) 2024-02-07T13:05:03.448 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-07T13:05:03.449 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T13:05:03.593 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '44458ccedb18241cc2e17f10cdab3d1385399261', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76537/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1252-g44458cce', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1252.g44458cce'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-guits-testing-4-2024-02-05-1854/44458ccedb18241cc2e17f10cdab3d1385399261/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-05 21:02:35.601617', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-guits-testing-4-2024-02-05-1854', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-guits-testing-4-2024-02-05-1854/44458ccedb18241cc2e17f10cdab3d1385399261/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-02-07T13:05:03.861 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-guits-testing-4-2024-02-05-1854, sha1 44458ccedb18241cc2e17f10cdab3d1385399261 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T13:05:03.863 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-guits-testing-4-2024-02-05-1854/44458ccedb18241cc2e17f10cdab3d1385399261/centos/9/x86_64/flavors/default/cephadm 2024-02-07T13:05:03.864 DEBUG:teuthology.orchestra.run.smithi067:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-guits-testing-4-2024-02-05-1854/44458ccedb18241cc2e17f10cdab3d1385399261/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-07T13:05:04.294 INFO:teuthology.orchestra.run.smithi067.stdout:-rw-r--r--. 1 ubuntu ubuntu 766878 Feb 7 13:05 /home/ubuntu/cephtest/cephadm 2024-02-07T13:05:04.295 DEBUG:teuthology.orchestra.run.smithi159:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-guits-testing-4-2024-02-05-1854/44458ccedb18241cc2e17f10cdab3d1385399261/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-07T13:05:05.199 INFO:teuthology.orchestra.run.smithi159.stdout:-rw-r--r--. 1 ubuntu ubuntu 766878 Feb 7 13:05 /home/ubuntu/cephtest/cephadm 2024-02-07T13:05:05.199 DEBUG:teuthology.orchestra.run.smithi191:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-guits-testing-4-2024-02-05-1854/44458ccedb18241cc2e17f10cdab3d1385399261/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-07T13:05:06.578 INFO:teuthology.orchestra.run.smithi191.stdout:-rw-r--r--. 1 ubuntu ubuntu 766878 Feb 7 13:05 /home/ubuntu/cephtest/cephadm 2024-02-07T13:05:06.579 DEBUG:teuthology.orchestra.run.smithi067:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-07T13:05:06.597 DEBUG:teuthology.orchestra.run.smithi159:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-07T13:05:06.615 DEBUG:teuthology.orchestra.run.smithi191:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-07T13:05:06.654 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 on all hosts... 2024-02-07T13:05:06.654 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 pull 2024-02-07T13:05:06.657 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 pull 2024-02-07T13:05:06.659 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 pull 2024-02-07T13:05:07.509 INFO:teuthology.orchestra.run.smithi067.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261... 2024-02-07T13:05:07.513 INFO:teuthology.orchestra.run.smithi159.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261... 2024-02-07T13:05:07.551 INFO:teuthology.orchestra.run.smithi191.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261... 2024-02-07T13:05:45.791 INFO:teuthology.orchestra.run.smithi191.stdout:{ 2024-02-07T13:05:45.791 INFO:teuthology.orchestra.run.smithi191.stdout: "ceph_version": "ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev)", 2024-02-07T13:05:45.791 INFO:teuthology.orchestra.run.smithi191.stdout: "image_id": "79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f", 2024-02-07T13:05:45.791 INFO:teuthology.orchestra.run.smithi191.stdout: "repo_digests": [ 2024-02-07T13:05:45.791 INFO:teuthology.orchestra.run.smithi191.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe" 2024-02-07T13:05:45.792 INFO:teuthology.orchestra.run.smithi191.stdout: ] 2024-02-07T13:05:45.792 INFO:teuthology.orchestra.run.smithi191.stdout:} 2024-02-07T13:05:52.053 INFO:teuthology.orchestra.run.smithi159.stdout:{ 2024-02-07T13:05:52.053 INFO:teuthology.orchestra.run.smithi159.stdout: "ceph_version": "ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev)", 2024-02-07T13:05:52.053 INFO:teuthology.orchestra.run.smithi159.stdout: "image_id": "79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f", 2024-02-07T13:05:52.053 INFO:teuthology.orchestra.run.smithi159.stdout: "repo_digests": [ 2024-02-07T13:05:52.053 INFO:teuthology.orchestra.run.smithi159.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe" 2024-02-07T13:05:52.053 INFO:teuthology.orchestra.run.smithi159.stdout: ] 2024-02-07T13:05:52.053 INFO:teuthology.orchestra.run.smithi159.stdout:} 2024-02-07T13:05:52.942 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-02-07T13:05:52.942 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev)", 2024-02-07T13:05:52.942 INFO:teuthology.orchestra.run.smithi067.stdout: "image_id": "79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f", 2024-02-07T13:05:52.942 INFO:teuthology.orchestra.run.smithi067.stdout: "repo_digests": [ 2024-02-07T13:05:52.942 INFO:teuthology.orchestra.run.smithi067.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe" 2024-02-07T13:05:52.942 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-02-07T13:05:52.943 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-02-07T13:05:52.967 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /etc/ceph 2024-02-07T13:05:53.015 DEBUG:teuthology.orchestra.run.smithi159:> sudo mkdir -p /etc/ceph 2024-02-07T13:05:53.050 DEBUG:teuthology.orchestra.run.smithi191:> sudo mkdir -p /etc/ceph 2024-02-07T13:05:53.086 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 777 /etc/ceph 2024-02-07T13:05:53.112 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 777 /etc/ceph 2024-02-07T13:05:53.143 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 777 /etc/ceph 2024-02-07T13:05:53.174 INFO:tasks.cephadm:Writing seed config... 2024-02-07T13:05:53.176 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-07T13:05:53.176 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-07T13:05:53.176 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = True 2024-02-07T13:05:53.176 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-02-07T13:05:53.176 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-07T13:05:53.176 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-07T13:05:53.176 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-07T13:05:53.176 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-07T13:05:53.177 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-07T13:05:53.177 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-07T13:05:53.177 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-07T13:05:53.177 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-07T13:05:53.192 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 81df3e5a-c5b9-11ee-95b6-87774f69a715 mon election default strategy = 1 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = True [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-02-07T13:05:53.192 DEBUG:teuthology.orchestra.run.smithi067:mon.a> sudo journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mon.a.service 2024-02-07T13:05:53.236 DEBUG:teuthology.orchestra.run.smithi067:mgr.y> sudo journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mgr.y.service 2024-02-07T13:05:53.279 INFO:tasks.cephadm:Bootstrapping... 2024-02-07T13:05:53.279 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 -v bootstrap --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.67 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-07T13:05:53.474 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-02-07T13:05:53.475 INFO:teuthology.orchestra.run.smithi067.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261', '-v', 'bootstrap', '--fsid', '81df3e5a-c5b9-11ee-95b6-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.67', '--skip-admin-label'] 2024-02-07T13:05:53.501 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 4.8.1 2024-02-07T13:05:53.502 INFO:teuthology.orchestra.run.smithi067.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-07T13:05:53.502 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying podman|docker is present... 2024-02-07T13:05:53.524 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 4.8.1 2024-02-07T13:05:53.524 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying lvm2 is present... 2024-02-07T13:05:53.525 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying time synchronization is in place... 2024-02-07T13:05:53.532 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-07T13:05:53.532 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-07T13:05:53.537 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-07T13:05:53.537 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-02-07T13:05:53.543 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout enabled 2024-02-07T13:05:53.550 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout active 2024-02-07T13:05:53.550 INFO:teuthology.orchestra.run.smithi067.stdout:Unit chronyd.service is enabled and running 2024-02-07T13:05:53.550 INFO:teuthology.orchestra.run.smithi067.stdout:Repeating the final host check... 2024-02-07T13:05:53.572 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 4.8.1 2024-02-07T13:05:53.573 INFO:teuthology.orchestra.run.smithi067.stdout:podman (/bin/podman) version 4.8.1 is present 2024-02-07T13:05:53.573 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl is present 2024-02-07T13:05:53.573 INFO:teuthology.orchestra.run.smithi067.stdout:lvcreate is present 2024-02-07T13:05:53.578 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-07T13:05:53.578 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-07T13:05:53.584 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-07T13:05:53.584 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-02-07T13:05:53.590 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout enabled 2024-02-07T13:05:53.596 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout active 2024-02-07T13:05:53.596 INFO:teuthology.orchestra.run.smithi067.stdout:Unit chronyd.service is enabled and running 2024-02-07T13:05:53.596 INFO:teuthology.orchestra.run.smithi067.stdout:Host looks OK 2024-02-07T13:05:53.597 INFO:teuthology.orchestra.run.smithi067.stdout:Cluster fsid: 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:05:53.597 INFO:teuthology.orchestra.run.smithi067.stdout:Acquiring lock 140520449508928 on /run/cephadm/81df3e5a-c5b9-11ee-95b6-87774f69a715.lock 2024-02-07T13:05:53.597 INFO:teuthology.orchestra.run.smithi067.stdout:Lock 140520449508928 acquired on /run/cephadm/81df3e5a-c5b9-11ee-95b6-87774f69a715.lock 2024-02-07T13:05:53.597 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying IP 172.21.15.67 port 3300 ... 2024-02-07T13:05:53.597 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying IP 172.21.15.67 port 6789 ... 2024-02-07T13:05:53.597 INFO:teuthology.orchestra.run.smithi067.stdout:Base mon IP(s) is [172.21.15.67:3300, 172.21.15.67:6789], mon addrv is [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2024-02-07T13:05:53.600 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.67 metric 100 2024-02-07T13:05:53.600 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.67 metric 100 2024-02-07T13:05:53.604 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-07T13:05:53.604 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-07T13:05:53.604 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1743sec hoplimit 64 pref medium 2024-02-07T13:05:53.606 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-07T13:05:53.606 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-07T13:05:53.606 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-07T13:05:53.606 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-07T13:05:53.606 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:feb6:fc0b/64 scope link 2024-02-07T13:05:53.606 INFO:teuthology.orchestra.run.smithi067.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-07T13:05:53.607 INFO:teuthology.orchestra.run.smithi067.stdout:Mon IP `172.21.15.67` is in CIDR network `172.21.0.0/20` 2024-02-07T13:05:53.607 INFO:teuthology.orchestra.run.smithi067.stdout:Mon IP `172.21.15.67` is in CIDR network `172.21.0.0/20` 2024-02-07T13:05:53.607 INFO:teuthology.orchestra.run.smithi067.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-07T13:05:53.607 INFO:teuthology.orchestra.run.smithi067.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-07T13:05:53.608 INFO:teuthology.orchestra.run.smithi067.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261... 2024-02-07T13:05:54.078 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stdout 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f 2024-02-07T13:05:54.078 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261... 2024-02-07T13:05:54.078 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Getting image source signatures 2024-02-07T13:05:54.078 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying blob sha256:a997f0e9fc346761d24d2a6e2e6c8569e51b56d4493dc8639d7f291d92935b5f 2024-02-07T13:05:54.078 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying blob sha256:58df4e03ba066accf3a7e07426bab5f1c867e27a139801c642bdafb42b9a2d9a 2024-02-07T13:05:54.078 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Copying config sha256:79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f 2024-02-07T13:05:54.078 INFO:teuthology.orchestra.run.smithi067.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-07T13:05:54.999 INFO:teuthology.orchestra.run.smithi067.stdout:ceph: stdout ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev) 2024-02-07T13:05:54.999 INFO:teuthology.orchestra.run.smithi067.stdout:Ceph version: ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev) 2024-02-07T13:05:54.999 INFO:teuthology.orchestra.run.smithi067.stdout:Extracting ceph user uid/gid from container image... 2024-02-07T13:05:55.895 INFO:teuthology.orchestra.run.smithi067.stdout:stat: stdout 167 167 2024-02-07T13:05:55.895 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial keys... 2024-02-07T13:05:56.898 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQA0gMNlcdYeDhAAnAwDMO7nE1m7AVckG+0Srg== 2024-02-07T13:05:57.886 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQA1gMNlLfJvDhAAE5GtA2zlBCn5OUieruo8fw== 2024-02-07T13:05:58.834 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph-authtool: stdout AQA2gMNls/pDDhAArUYuESAIevGJfrKjQdu/ng== 2024-02-07T13:05:58.835 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial monmap... 2024-02-07T13:06:06.115 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-07T13:06:06.115 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-07T13:06:06.115 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:06:06.115 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-07T13:06:06.115 INFO:teuthology.orchestra.run.smithi067.stdout:monmaptool for a [v2:172.21.15.67:3300,v1:172.21.15.67:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-07T13:06:06.115 INFO:teuthology.orchestra.run.smithi067.stdout:setting min_mon_release = quincy 2024-02-07T13:06:06.115 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: set fsid to 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:06:06.116 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-07T13:06:06.116 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:06:06.116 INFO:teuthology.orchestra.run.smithi067.stdout:Creating mon... 2024-02-07T13:06:10.145 INFO:teuthology.orchestra.run.smithi067.stdout:create mon.a on 2024-02-07T13:06:10.515 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-07T13:06:10.693 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715.target → /etc/systemd/system/ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715.target. 2024-02-07T13:06:10.693 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715.target → /etc/systemd/system/ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715.target. 2024-02-07T13:06:10.941 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mon.a 2024-02-07T13:06:10.941 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to reset failed state of unit ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mon.a.service: Unit ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mon.a.service not loaded. 2024-02-07T13:06:11.107 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715.target.wants/ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mon.a.service → /etc/systemd/system/ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@.service. 2024-02-07T13:06:11.438 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:11 smithi067 podman[25171]: 2024-02-07 13:06:11.241089369 +0000 UTC m=+0.018185544 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T13:06:11.594 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-07T13:06:11.594 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-02-07T13:06:11.601 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-07T13:06:11.601 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-02-07T13:06:11.601 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-02-07T13:06:11.601 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to enable service . firewalld.service is not available 2024-02-07T13:06:11.601 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mon to start... 2024-02-07T13:06:11.602 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mon... 2024-02-07T13:06:11.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:11 smithi067 podman[25171]: 2024-02-07 13:06:11.513117014 +0000 UTC m=+0.290213189 container init e3efae0bca6e507caf08d9795d12b94069a60837fc38e64a16ea8643e8fc9429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-a, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-02-07T13:06:11.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:11 smithi067 podman[25171]: 2024-02-07 13:06:11.519299022 +0000 UTC m=+0.296395202 container start e3efae0bca6e507caf08d9795d12b94069a60837fc38e64a16ea8643e8fc9429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-07T13:06:11.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:11 smithi067 bash[25171]: e3efae0bca6e507caf08d9795d12b94069a60837fc38e64a16ea8643e8fc9429 2024-02-07T13:06:11.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:11 smithi067 systemd[1]: Started Ceph mon.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:06:11.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:11 smithi067 ceph-mon[25209]: mkfs 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:06:11.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:11 smithi067 ceph-mon[25209]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-07T13:06:12.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:12 smithi067 ceph-mon[25209]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-07T13:06:12.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:12 smithi067 ceph-mon[25209]: monmap e1: 1 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T13:06:12.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:12 smithi067 ceph-mon[25209]: fsmap 2024-02-07T13:06:12.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:12 smithi067 ceph-mon[25209]: osdmap e1: 0 total, 0 up, 0 in 2024-02-07T13:06:12.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:12 smithi067 ceph-mon[25209]: mgrmap e1: no daemons active 2024-02-07T13:06:12.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:12 smithi067 ceph-mon[25209]: from='client.? 172.21.15.67:0/58180882' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-07T13:06:12.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout cluster: 2024-02-07T13:06:12.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout id: 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:06:12.765 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-07T13:06:12.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-02-07T13:06:12.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout services: 2024-02-07T13:06:12.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.577287s) 2024-02-07T13:06:12.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-07T13:06:12.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-07T13:06:12.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-02-07T13:06:12.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout data: 2024-02-07T13:06:12.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-07T13:06:12.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-07T13:06:12.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-07T13:06:12.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout pgs: 2024-02-07T13:06:12.766 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-02-07T13:06:12.766 INFO:teuthology.orchestra.run.smithi067.stdout:mon is available 2024-02-07T13:06:12.766 INFO:teuthology.orchestra.run.smithi067.stdout:Assimilating anything we can from ceph.conf... 2024-02-07T13:06:14.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:13 smithi067 ceph-mon[25209]: from='client.? 172.21.15.67:0/2057714138' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-07T13:06:14.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:13 smithi067 ceph-mon[25209]: from='client.? 172.21.15.67:0/2057714138' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-07T13:06:14.105 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-02-07T13:06:14.105 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [global] 2024-02-07T13:06:14.105 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout fsid = 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:06:14.105 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.67:3300,v1:172.21.15.67:6789] 2024-02-07T13:06:14.105 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-07T13:06:14.105 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-07T13:06:14.105 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-07T13:06:14.105 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-07T13:06:14.105 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-02-07T13:06:14.105 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-07T13:06:14.106 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = True 2024-02-07T13:06:14.106 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-07T13:06:14.106 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-02-07T13:06:14.106 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [osd] 2024-02-07T13:06:14.106 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-07T13:06:14.106 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-07T13:06:14.106 INFO:teuthology.orchestra.run.smithi067.stdout:Generating new minimal ceph.conf... 2024-02-07T13:06:14.695 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:14 smithi067 ceph-mon[25209]: from='client.? 172.21.15.67:0/2016843188' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:06:16.761 INFO:teuthology.orchestra.run.smithi067.stdout:Restarting the monitor... 2024-02-07T13:06:17.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:16 smithi067 systemd[1]: Stopping Ceph mon.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:06:17.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:17 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-a[25186]: 2024-02-07T13:06:17.596+0000 7f857867d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T13:06:17.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:17 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-a[25186]: 2024-02-07T13:06:17.596+0000 7f857867d700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-07T13:06:17.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:17 smithi067 podman[25540]: 2024-02-07 13:06:17.617663471 +0000 UTC m=+0.676142484 container died e3efae0bca6e507caf08d9795d12b94069a60837fc38e64a16ea8643e8fc9429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-a, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-07T13:06:18.152 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:17 smithi067 podman[25540]: 2024-02-07 13:06:17.951134041 +0000 UTC m=+1.009613053 container cleanup e3efae0bca6e507caf08d9795d12b94069a60837fc38e64a16ea8643e8fc9429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-a, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-07T13:06:18.152 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:17 smithi067 bash[25540]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-a 2024-02-07T13:06:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 podman[25559]: 2024-02-07 13:06:18.151904099 +0000 UTC m=+0.526610985 container remove e3efae0bca6e507caf08d9795d12b94069a60837fc38e64a16ea8643e8fc9429 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.2) 2024-02-07T13:06:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-07T13:06:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 systemd[1]: Stopped Ceph mon.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:06:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 systemd[1]: Starting Ceph mon.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:06:18.855 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 podman[25660]: 2024-02-07 13:06:18.50110337 +0000 UTC m=+0.018973562 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T13:06:18.856 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 podman[25660]: 2024-02-07 13:06:18.602684686 +0000 UTC m=+0.120554880 container create 4aae939770198905808bc85f38a03e2a5d433e80eca80b01a9871444345a6c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-a, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131) 2024-02-07T13:06:18.856 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 podman[25660]: 2024-02-07 13:06:18.76394989 +0000 UTC m=+0.281820099 container init 4aae939770198905808bc85f38a03e2a5d433e80eca80b01a9871444345a6c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131) 2024-02-07T13:06:18.856 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 podman[25660]: 2024-02-07 13:06:18.769637271 +0000 UTC m=+0.287507465 container start 4aae939770198905808bc85f38a03e2a5d433e80eca80b01a9871444345a6c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-07T13:06:18.856 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: set uid:gid to 167:167 (ceph:ceph) 2024-02-07T13:06:18.856 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev), process ceph-mon, pid 7 2024-02-07T13:06:18.856 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: pidfile_write: ignore empty --pid-file 2024-02-07T13:06:18.856 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: load: jerasure load: lrc 2024-02-07T13:06:18.856 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: RocksDB version: 7.9.2 2024-02-07T13:06:18.856 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Git sha 0 2024-02-07T13:06:18.856 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Compile date 2024-02-05 19:41:51 2024-02-07T13:06:18.857 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: DB SUMMARY 2024-02-07T13:06:18.857 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: DB Session ID: ZBODU1OOBYMTJXP8TXH0 2024-02-07T13:06:18.857 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: CURRENT file: CURRENT 2024-02-07T13:06:18.857 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: IDENTITY file: IDENTITY 2024-02-07T13:06:18.857 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-07T13:06:18.857 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-07T13:06:18.857 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 85122 ; 2024-02-07T13:06:18.857 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.error_if_exists: 0 2024-02-07T13:06:18.857 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.create_if_missing: 0 2024-02-07T13:06:18.857 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.paranoid_checks: 1 2024-02-07T13:06:18.857 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-07T13:06:18.857 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-07T13:06:18.857 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-07T13:06:18.857 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.env: 0x55cdeeddaae0 2024-02-07T13:06:18.858 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.fs: PosixFileSystem 2024-02-07T13:06:18.858 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.info_log: 0x55cdf0dd38c0 2024-02-07T13:06:18.858 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_file_opening_threads: 16 2024-02-07T13:06:18.858 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.statistics: (nil) 2024-02-07T13:06:18.858 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.use_fsync: 0 2024-02-07T13:06:18.858 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_log_file_size: 0 2024-02-07T13:06:18.858 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-07T13:06:18.858 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-07T13:06:18.858 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.keep_log_file_num: 1000 2024-02-07T13:06:18.858 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.recycle_log_file_num: 0 2024-02-07T13:06:18.858 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.allow_fallocate: 1 2024-02-07T13:06:18.858 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.allow_mmap_reads: 0 2024-02-07T13:06:18.858 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.allow_mmap_writes: 0 2024-02-07T13:06:18.859 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.use_direct_reads: 0 2024-02-07T13:06:18.859 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-07T13:06:18.859 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.create_missing_column_families: 0 2024-02-07T13:06:18.859 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.db_log_dir: 2024-02-07T13:06:18.859 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.wal_dir: 2024-02-07T13:06:18.859 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-07T13:06:18.859 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-07T13:06:18.859 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-07T13:06:18.859 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-07T13:06:18.859 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-07T13:06:18.859 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-07T13:06:18.859 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.advise_random_on_open: 1 2024-02-07T13:06:18.859 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.db_write_buffer_size: 0 2024-02-07T13:06:18.859 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.write_buffer_manager: 0x55cdf0da7400 2024-02-07T13:06:18.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-07T13:06:18.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-07T13:06:18.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-07T13:06:18.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.rate_limiter: (nil) 2024-02-07T13:06:18.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-07T13:06:18.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.wal_recovery_mode: 2 2024-02-07T13:06:18.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.enable_thread_tracking: 0 2024-02-07T13:06:18.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.enable_pipelined_write: 0 2024-02-07T13:06:18.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.unordered_write: 0 2024-02-07T13:06:18.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-07T13:06:18.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-07T13:06:18.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-07T13:06:18.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-07T13:06:18.860 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.row_cache: None 2024-02-07T13:06:18.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.wal_filter: None 2024-02-07T13:06:18.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-07T13:06:18.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.allow_ingest_behind: 0 2024-02-07T13:06:18.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.two_write_queues: 0 2024-02-07T13:06:18.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.manual_wal_flush: 0 2024-02-07T13:06:18.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.wal_compression: 0 2024-02-07T13:06:18.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.atomic_flush: 0 2024-02-07T13:06:18.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-07T13:06:18.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-07T13:06:18.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-07T13:06:18.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.log_readahead_size: 0 2024-02-07T13:06:18.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-07T13:06:18.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.best_efforts_recovery: 0 2024-02-07T13:06:18.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-07T13:06:18.861 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-07T13:06:18.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.allow_data_in_errors: 0 2024-02-07T13:06:18.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.db_host_id: __hostname__ 2024-02-07T13:06:18.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-07T13:06:18.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_background_jobs: 2 2024-02-07T13:06:18.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_background_compactions: -1 2024-02-07T13:06:18.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_subcompactions: 1 2024-02-07T13:06:18.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-07T13:06:18.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-07T13:06:18.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-07T13:06:18.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_total_wal_size: 0 2024-02-07T13:06:18.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-07T13:06:18.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-07T13:06:18.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-07T13:06:18.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-07T13:06:18.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_open_files: -1 2024-02-07T13:06:18.862 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.bytes_per_sync: 0 2024-02-07T13:06:18.864 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-07T13:06:18.864 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-07T13:06:18.864 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compaction_readahead_size: 0 2024-02-07T13:06:18.864 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_background_flushes: -1 2024-02-07T13:06:18.864 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Compression algorithms supported: 2024-02-07T13:06:18.864 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-07T13:06:18.864 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: kZSTD supported: 0 2024-02-07T13:06:18.864 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: kXpressCompression supported: 0 2024-02-07T13:06:18.864 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: kLZ4HCCompression supported: 1 2024-02-07T13:06:18.864 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: kZlibCompression supported: 1 2024-02-07T13:06:18.865 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: kSnappyCompression supported: 1 2024-02-07T13:06:18.865 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: kLZ4Compression supported: 1 2024-02-07T13:06:18.865 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: kBZip2Compression supported: 0 2024-02-07T13:06:18.865 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-07T13:06:18.865 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-07T13:06:18.865 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-07T13:06:18.865 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-07T13:06:18.865 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-07T13:06:18.865 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.merge_operator: 2024-02-07T13:06:18.865 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compaction_filter: None 2024-02-07T13:06:18.865 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compaction_filter_factory: None 2024-02-07T13:06:18.865 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.sst_partitioner_factory: None 2024-02-07T13:06:18.865 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-07T13:06:18.865 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-07T13:06:18.865 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cdf0dd39c0) 2024-02-07T13:06:18.866 INFO:journalctl@ceph.mon.a.smithi067.stdout: cache_index_and_filter_blocks: 1 2024-02-07T13:06:18.866 INFO:journalctl@ceph.mon.a.smithi067.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-07T13:06:18.866 INFO:journalctl@ceph.mon.a.smithi067.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-07T13:06:18.866 INFO:journalctl@ceph.mon.a.smithi067.stdout: pin_top_level_index_and_filter: 1 2024-02-07T13:06:18.866 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_type: 0 2024-02-07T13:06:18.866 INFO:journalctl@ceph.mon.a.smithi067.stdout: data_block_index_type: 0 2024-02-07T13:06:18.866 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_shortening: 1 2024-02-07T13:06:18.866 INFO:journalctl@ceph.mon.a.smithi067.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-07T13:06:18.866 INFO:journalctl@ceph.mon.a.smithi067.stdout: checksum: 4 2024-02-07T13:06:18.866 INFO:journalctl@ceph.mon.a.smithi067.stdout: no_block_cache: 0 2024-02-07T13:06:18.866 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache: 0x55cdf0dd7090 2024-02-07T13:06:18.866 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_name: BinnedLRUCache 2024-02-07T13:06:18.866 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_options: 2024-02-07T13:06:18.866 INFO:journalctl@ceph.mon.a.smithi067.stdout: capacity : 536870912 2024-02-07T13:06:18.866 INFO:journalctl@ceph.mon.a.smithi067.stdout: num_shard_bits : 4 2024-02-07T13:06:18.867 INFO:journalctl@ceph.mon.a.smithi067.stdout: strict_capacity_limit : 0 2024-02-07T13:06:18.867 INFO:journalctl@ceph.mon.a.smithi067.stdout: high_pri_pool_ratio: 0.000 2024-02-07T13:06:18.867 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_cache_compressed: (nil) 2024-02-07T13:06:18.867 INFO:journalctl@ceph.mon.a.smithi067.stdout: persistent_cache: (nil) 2024-02-07T13:06:18.867 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_size: 4096 2024-02-07T13:06:18.867 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_size_deviation: 10 2024-02-07T13:06:18.867 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_restart_interval: 16 2024-02-07T13:06:18.867 INFO:journalctl@ceph.mon.a.smithi067.stdout: index_block_restart_interval: 1 2024-02-07T13:06:18.867 INFO:journalctl@ceph.mon.a.smithi067.stdout: metadata_block_size: 4096 2024-02-07T13:06:18.867 INFO:journalctl@ceph.mon.a.smithi067.stdout: partition_filters: 0 2024-02-07T13:06:18.867 INFO:journalctl@ceph.mon.a.smithi067.stdout: use_delta_encoding: 1 2024-02-07T13:06:18.867 INFO:journalctl@ceph.mon.a.smithi067.stdout: filter_policy: bloomfilter 2024-02-07T13:06:18.867 INFO:journalctl@ceph.mon.a.smithi067.stdout: whole_key_filtering: 1 2024-02-07T13:06:18.867 INFO:journalctl@ceph.mon.a.smithi067.stdout: verify_compression: 0 2024-02-07T13:06:18.867 INFO:journalctl@ceph.mon.a.smithi067.stdout: read_amp_bytes_per_bit: 0 2024-02-07T13:06:18.867 INFO:journalctl@ceph.mon.a.smithi067.stdout: format_version: 5 2024-02-07T13:06:18.867 INFO:journalctl@ceph.mon.a.smithi067.stdout: enable_index_compression: 1 2024-02-07T13:06:18.868 INFO:journalctl@ceph.mon.a.smithi067.stdout: block_align: 0 2024-02-07T13:06:18.868 INFO:journalctl@ceph.mon.a.smithi067.stdout: max_auto_readahead_size: 262144 2024-02-07T13:06:18.868 INFO:journalctl@ceph.mon.a.smithi067.stdout: prepopulate_block_cache: 0 2024-02-07T13:06:18.868 INFO:journalctl@ceph.mon.a.smithi067.stdout: initial_auto_readahead_size: 8192 2024-02-07T13:06:18.868 INFO:journalctl@ceph.mon.a.smithi067.stdout: num_file_reads_for_auto_readahead: 2 2024-02-07T13:06:18.868 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.write_buffer_size: 33554432 2024-02-07T13:06:18.868 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_write_buffer_number: 2 2024-02-07T13:06:18.868 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compression: NoCompression 2024-02-07T13:06:18.868 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.bottommost_compression: Disabled 2024-02-07T13:06:18.868 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.prefix_extractor: nullptr 2024-02-07T13:06:18.868 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-07T13:06:18.868 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.num_levels: 7 2024-02-07T13:06:18.868 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-07T13:06:18.868 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-07T13:06:18.868 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-07T13:06:18.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-07T13:06:18.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-07T13:06:18.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-07T13:06:18.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-07T13:06:18.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-07T13:06:18.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-07T13:06:18.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-07T13:06:18.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-07T13:06:18.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-07T13:06:18.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-07T13:06:18.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compression_opts.level: 32767 2024-02-07T13:06:18.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compression_opts.strategy: 0 2024-02-07T13:06:18.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-07T13:06:18.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-07T13:06:18.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-07T13:06:18.869 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-07T13:06:18.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compression_opts.enabled: false 2024-02-07T13:06:18.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-07T13:06:18.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-07T13:06:18.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-07T13:06:18.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-07T13:06:18.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.target_file_size_base: 67108864 2024-02-07T13:06:18.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-07T13:06:18.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-07T13:06:18.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-07T13:06:18.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-07T13:06:18.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-07T13:06:18.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-07T13:06:18.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-07T13:06:18.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-07T13:06:18.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-07T13:06:18.870 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-07T13:06:18.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-07T13:06:18.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-07T13:06:18.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-07T13:06:18.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-07T13:06:18.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.arena_block_size: 1048576 2024-02-07T13:06:18.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-07T13:06:18.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-07T13:06:18.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.disable_auto_compactions: 0 2024-02-07T13:06:18.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-07T13:06:18.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-07T13:06:18.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-07T13:06:18.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-07T13:06:18.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-07T13:06:18.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-07T13:06:18.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-07T13:06:18.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-07T13:06:18.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-07T13:06:18.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-07T13:06:18.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.table_properties_collectors: 2024-02-07T13:06:18.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.inplace_update_support: 0 2024-02-07T13:06:18.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-07T13:06:18.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-07T13:06:18.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-07T13:06:18.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-07T13:06:18.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.bloom_locality: 0 2024-02-07T13:06:18.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.max_successive_merges: 0 2024-02-07T13:06:18.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-07T13:06:18.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.paranoid_file_checks: 0 2024-02-07T13:06:18.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.force_consistency_checks: 1 2024-02-07T13:06:18.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.report_bg_io_stats: 0 2024-02-07T13:06:18.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.ttl: 2592000 2024-02-07T13:06:18.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-07T13:06:18.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-07T13:06:18.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-07T13:06:18.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.enable_blob_files: false 2024-02-07T13:06:18.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.min_blob_size: 0 2024-02-07T13:06:18.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.blob_file_size: 268435456 2024-02-07T13:06:18.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-07T13:06:18.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-07T13:06:18.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-07T13:06:18.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-07T13:06:18.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-07T13:06:18.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.blob_file_starting_level: 0 2024-02-07T13:06:18.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-07T13:06:18.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-07T13:06:18.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-07T13:06:18.873 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: ee143710-1cda-495e-a9be-2ec5fddb8c48 2024-02-07T13:06:18.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707311178797844, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-07T13:06:18.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-07T13:06:18.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707311178798673, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 81503, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 261, "table_properties": {"data_size": 79690, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10043, "raw_average_key_size": 47, "raw_value_size": 73986, "raw_average_value_size": 348, "num_data_blocks": 10, "num_entries": 212, "num_filter_entries": 212, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707311178, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "ee143710-1cda-495e-a9be-2ec5fddb8c48", "db_session_id": "ZBODU1OOBYMTJXP8TXH0", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-07T13:06:18.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707311178798755, "job": 1, "event": "recovery_finished"} 2024-02-07T13:06:18.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-07T13:06:18.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-07T13:06:18.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cdf0ec8000 2024-02-07T13:06:18.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: DB pointer 0x55cdf0eb2000 2024-02-07T13:06:18.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-07T13:06:18.874 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-07T13:06:18.874 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** DB Stats ** 2024-02-07T13:06:18.874 INFO:journalctl@ceph.mon.a.smithi067.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-07T13:06:18.874 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-07T13:06:18.874 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-07T13:06:18.874 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-07T13:06:18.875 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-07T13:06:18.875 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-07T13:06:18.875 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-07T13:06:18.875 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-02-07T13:06:18.875 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** Compaction Stats [default] ** 2024-02-07T13:06:18.875 INFO:journalctl@ceph.mon.a.smithi067.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-07T13:06:18.875 INFO:journalctl@ceph.mon.a.smithi067.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-07T13:06:18.875 INFO:journalctl@ceph.mon.a.smithi067.stdout: L0 2/0 81.43 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-07T13:06:18.875 INFO:journalctl@ceph.mon.a.smithi067.stdout: Sum 2/0 81.43 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-07T13:06:18.875 INFO:journalctl@ceph.mon.a.smithi067.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-07T13:06:18.875 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-02-07T13:06:18.875 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** Compaction Stats [default] ** 2024-02-07T13:06:18.875 INFO:journalctl@ceph.mon.a.smithi067.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-07T13:06:18.875 INFO:journalctl@ceph.mon.a.smithi067.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-07T13:06:18.875 INFO:journalctl@ceph.mon.a.smithi067.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 151.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-07T13:06:18.876 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-02-07T13:06:18.876 INFO:journalctl@ceph.mon.a.smithi067.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-07T13:06:18.876 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-02-07T13:06:18.876 INFO:journalctl@ceph.mon.a.smithi067.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-07T13:06:18.876 INFO:journalctl@ceph.mon.a.smithi067.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-07T13:06:18.876 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-07T13:06:18.876 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-07T13:06:18.876 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-07T13:06:18.876 INFO:journalctl@ceph.mon.a.smithi067.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-07T13:06:18.876 INFO:journalctl@ceph.mon.a.smithi067.stdout: Cumulative compaction: 0.00 GB write, 27.94 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-07T13:06:18.876 INFO:journalctl@ceph.mon.a.smithi067.stdout: Interval compaction: 0.00 GB write, 27.94 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-07T13:06:18.876 INFO:journalctl@ceph.mon.a.smithi067.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-07T13:06:18.876 INFO:journalctl@ceph.mon.a.smithi067.stdout: Block cache BinnedLRUCache@0x55cdf0dd7090#7 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-02-07T13:06:18.876 INFO:journalctl@ceph.mon.a.smithi067.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-02-07T13:06:18.876 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-02-07T13:06:18.877 INFO:journalctl@ceph.mon.a.smithi067.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-07T13:06:18.877 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: starting mon.a rank 0 at public addrs [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] at bind addrs [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:06:18.877 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: mon.a@-1(???) e1 preinit fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:06:18.877 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: mon.a@-1(???).mds e1 new map 2024-02-07T13:06:18.877 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: mon.a@-1(???).mds e1 print_map 2024-02-07T13:06:18.877 INFO:journalctl@ceph.mon.a.smithi067.stdout: e1 2024-02-07T13:06:18.877 INFO:journalctl@ceph.mon.a.smithi067.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-07T13:06:18.877 INFO:journalctl@ceph.mon.a.smithi067.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-07T13:06:18.877 INFO:journalctl@ceph.mon.a.smithi067.stdout: legacy client fscid: -1 2024-02-07T13:06:18.877 INFO:journalctl@ceph.mon.a.smithi067.stdout: 2024-02-07T13:06:18.877 INFO:journalctl@ceph.mon.a.smithi067.stdout: No filesystems configured 2024-02-07T13:06:18.877 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-07T13:06:18.877 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T13:06:18.877 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T13:06:18.877 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T13:06:18.878 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-07T13:06:18.878 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 bash[25660]: 4aae939770198905808bc85f38a03e2a5d433e80eca80b01a9871444345a6c8c 2024-02-07T13:06:18.878 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 systemd[1]: Started Ceph mon.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:06:18.878 INFO:teuthology.orchestra.run.smithi067.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-07T13:06:19.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-07T13:06:19.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: monmap e1: 1 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T13:06:19.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: fsmap 2024-02-07T13:06:19.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: osdmap e1: 0 total, 0 up, 0 in 2024-02-07T13:06:19.167 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:18 smithi067 ceph-mon[25698]: mgrmap e1: no daemons active 2024-02-07T13:06:20.146 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-07T13:06:20.169 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-07T13:06:20.169 INFO:teuthology.orchestra.run.smithi067.stdout:Creating mgr... 2024-02-07T13:06:20.170 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-07T13:06:20.170 INFO:teuthology.orchestra.run.smithi067.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-07T13:06:20.420 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mgr.y 2024-02-07T13:06:20.421 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Failed to reset failed state of unit ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mgr.y.service: Unit ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mgr.y.service not loaded. 2024-02-07T13:06:20.576 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715.target.wants/ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@.service. 2024-02-07T13:06:20.920 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:20 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/4185087347' entity='client.admin' 2024-02-07T13:06:20.920 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:20 smithi067 podman[25940]: 2024-02-07 13:06:20.711130147 +0000 UTC m=+0.017221541 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 2024-02-07T13:06:21.137 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-07T13:06:21.138 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-02-07T13:06:21.145 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-07T13:06:21.145 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-02-07T13:06:21.145 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-02-07T13:06:21.145 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to enable service . firewalld.service is not available 2024-02-07T13:06:21.153 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-07T13:06:21.153 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-02-07T13:06:21.160 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-07T13:06:21.160 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-02-07T13:06:21.161 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-02-07T13:06:21.161 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-07T13:06:21.161 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr to start... 2024-02-07T13:06:21.161 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr... 2024-02-07T13:06:21.275 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:21 smithi067 podman[25940]: 2024-02-07 13:06:21.015161514 +0000 UTC m=+0.321252901 container init c67bb53a16c70efa315f9e094a7c58fbf35e3b7db8a13f79904fb5add555d662 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, CEPH_POINT_RELEASE=) 2024-02-07T13:06:21.275 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:21 smithi067 podman[25940]: 2024-02-07 13:06:21.021043573 +0000 UTC m=+0.327134963 container start c67bb53a16c70efa315f9e094a7c58fbf35e3b7db8a13f79904fb5add555d662 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y, GIT_CLEAN=True, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-07T13:06:21.275 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:21 smithi067 bash[25940]: c67bb53a16c70efa315f9e094a7c58fbf35e3b7db8a13f79904fb5add555d662 2024-02-07T13:06:21.275 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:21 smithi067 systemd[1]: Started Ceph mgr.y for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:06:21.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:21 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/1064833906' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-07T13:06:21.806 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:21 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:21.586+0000 7fab19b4f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T13:06:21.806 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:21 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:21.683+0000 7fab19b4f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T13:06:22.295 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-02-07T13:06:22.296 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-02-07T13:06:22.296 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "81df3e5a-c5b9-11ee-95b6-87774f69a715", 2024-02-07T13:06:22.296 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-02-07T13:06:22.296 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-07T13:06:22.296 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-07T13:06:22.296 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-07T13:06:22.296 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:22.296 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-07T13:06:22.296 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-07T13:06:22.296 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-02-07T13:06:22.296 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:06:22.296 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-07T13:06:22.296 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-02-07T13:06:22.297 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:06:22.297 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-07T13:06:22.297 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-07T13:06:22.297 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:06:22.297 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-07T13:06:22.297 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-07T13:06:22.297 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:22.297 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-07T13:06:22.297 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:06:22.297 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-07T13:06:22.297 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-07T13:06:22.297 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-07T13:06:22.297 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-07T13:06:22.297 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-07T13:06:22.298 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-07T13:06:22.298 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:22.298 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-07T13:06:22.298 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-07T13:06:22.298 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-07T13:06:22.298 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-07T13:06:22.298 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-07T13:06:22.298 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-07T13:06:22.298 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-07T13:06:22.298 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-07T13:06:22.298 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-07T13:06:22.298 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:22.298 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-07T13:06:22.298 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:06:22.299 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-07T13:06:22.299 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-07T13:06:22.299 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:22.299 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-07T13:06:22.299 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-07T13:06:22.299 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-07T13:06:22.299 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-07T13:06:22.299 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-07T13:06:22.299 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-07T13:06:22.299 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-02-07T13:06:22.299 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:06:22.299 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T13:06:22.299 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:22.299 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-07T13:06:22.299 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:06:22.299 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-02-07T13:06:11.589778+0000", 2024-02-07T13:06:22.299 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T13:06:22.300 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:22.300 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-07T13:06:22.300 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-02-07T13:06:22.300 INFO:teuthology.orchestra.run.smithi067.stdout:mgr not available, waiting (1/15)... 2024-02-07T13:06:22.684 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:22 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:22.351+0000 7fab19b4f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T13:06:22.684 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:22 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:22.444+0000 7fab19b4f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T13:06:22.684 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:22 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:22.586+0000 7fab19b4f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T13:06:22.685 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:22 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:22.683+0000 7fab19b4f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T13:06:23.032 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:22 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:22.766+0000 7fab19b4f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T13:06:23.032 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:23 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:23.031+0000 7fab19b4f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T13:06:23.306 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:23 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:23.142+0000 7fab19b4f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T13:06:23.306 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:23 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:23.228+0000 7fab19b4f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T13:06:23.657 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:23 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:23.324+0000 7fab19b4f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T13:06:23.657 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:23 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:23.501+0000 7fab19b4f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T13:06:24.789 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:24 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:24.514+0000 7fab19b4f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T13:06:24.789 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:24 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:24.648+0000 7fab19b4f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T13:06:25.056 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:24 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:24.996+0000 7fab19b4f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T13:06:25.347 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:25 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:25.092+0000 7fab19b4f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T13:06:25.347 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:25 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:25.293+0000 7fab19b4f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:06:25.686 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:25 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/3138572277' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-07T13:06:25.686 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:25 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:25.483+0000 7fab19b4f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T13:06:25.953 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:25 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:25.775+0000 7fab19b4f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:06:25.957 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-02-07T13:06:25.957 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-02-07T13:06:25.957 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "81df3e5a-c5b9-11ee-95b6-87774f69a715", 2024-02-07T13:06:25.957 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-02-07T13:06:25.957 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-07T13:06:25.957 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-07T13:06:25.957 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-07T13:06:25.957 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:25.957 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-07T13:06:25.957 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-07T13:06:25.957 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-02-07T13:06:25.958 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:06:25.958 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-07T13:06:25.958 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-02-07T13:06:25.958 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:06:25.958 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-02-07T13:06:25.958 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-07T13:06:25.958 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:06:25.958 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-07T13:06:25.958 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-07T13:06:25.958 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:25.958 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-07T13:06:25.958 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:06:25.958 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-07T13:06:25.958 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-07T13:06:25.958 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-07T13:06:25.958 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-07T13:06:25.959 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-07T13:06:25.959 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-07T13:06:25.962 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:25.962 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-07T13:06:25.962 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-07T13:06:25.962 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-07T13:06:25.962 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-07T13:06:25.962 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-07T13:06:25.962 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-07T13:06:25.963 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-07T13:06:25.963 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-07T13:06:25.963 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-07T13:06:25.963 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:25.963 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-07T13:06:25.963 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:06:25.963 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-07T13:06:25.963 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-07T13:06:25.963 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:25.963 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-07T13:06:25.963 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-07T13:06:25.963 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-07T13:06:25.963 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-07T13:06:25.963 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-07T13:06:25.963 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-07T13:06:25.964 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-02-07T13:06:25.964 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:06:25.964 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T13:06:25.964 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:25.964 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-07T13:06:25.964 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:06:25.964 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-02-07T13:06:11.589778+0000", 2024-02-07T13:06:25.964 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T13:06:25.964 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:25.964 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-07T13:06:25.964 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-02-07T13:06:25.964 INFO:teuthology.orchestra.run.smithi067.stdout:mgr not available, waiting (2/15)... 2024-02-07T13:06:26.277 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:26 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:26.037+0000 7fab19b4f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T13:06:26.297 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:26 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:26.275+0000 7fab19b4f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T13:06:27.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:27 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:27.092+0000 7fab19b4f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T13:06:27.989 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:27 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:27.669+0000 7fab19b4f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T13:06:27.989 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:27 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:27.768+0000 7fab19b4f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T13:06:27.989 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:27 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:27.987+0000 7fab19b4f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T13:06:28.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:28 smithi067 ceph-mon[25698]: Activating manager daemon y 2024-02-07T13:06:28.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:28 smithi067 ceph-mon[25698]: mgrmap e2: y(active, starting, since 0.0029038s) 2024-02-07T13:06:28.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:28 smithi067 ceph-mon[25698]: from='mgr.14100 172.21.15.67:0/3282291087' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-07T13:06:28.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:28 smithi067 ceph-mon[25698]: from='mgr.14100 172.21.15.67:0/3282291087' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-07T13:06:28.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:28 smithi067 ceph-mon[25698]: from='mgr.14100 172.21.15.67:0/3282291087' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-07T13:06:28.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:28 smithi067 ceph-mon[25698]: from='mgr.14100 172.21.15.67:0/3282291087' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:06:28.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:28 smithi067 ceph-mon[25698]: from='mgr.14100 172.21.15.67:0/3282291087' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-07T13:06:28.248 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:28 smithi067 ceph-mon[25698]: Manager daemon y is now available 2024-02-07T13:06:28.248 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:28 smithi067 ceph-mon[25698]: from='mgr.14100 172.21.15.67:0/3282291087' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-07T13:06:28.248 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:28 smithi067 ceph-mon[25698]: from='mgr.14100 172.21.15.67:0/3282291087' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-07T13:06:28.248 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:28 smithi067 ceph-mon[25698]: from='mgr.14100 172.21.15.67:0/3282291087' entity='mgr.y' 2024-02-07T13:06:28.248 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:28 smithi067 ceph-mon[25698]: from='mgr.14100 172.21.15.67:0/3282291087' entity='mgr.y' 2024-02-07T13:06:28.248 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:28 smithi067 ceph-mon[25698]: from='mgr.14100 172.21.15.67:0/3282291087' entity='mgr.y' 2024-02-07T13:06:29.233 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-02-07T13:06:29.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-02-07T13:06:29.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "81df3e5a-c5b9-11ee-95b6-87774f69a715", 2024-02-07T13:06:29.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-02-07T13:06:29.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-07T13:06:29.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-07T13:06:29.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-07T13:06:29.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:29.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-07T13:06:29.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-07T13:06:29.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-02-07T13:06:29.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:06:29.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-07T13:06:29.234 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-02-07T13:06:29.235 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:06:29.235 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-02-07T13:06:29.235 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-07T13:06:29.235 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:06:29.235 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-07T13:06:29.235 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-07T13:06:29.235 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:29.235 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-07T13:06:29.235 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:06:29.235 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-07T13:06:29.235 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-07T13:06:29.235 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-07T13:06:29.235 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-07T13:06:29.235 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-07T13:06:29.235 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-07T13:06:29.236 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:29.236 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-07T13:06:29.236 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-07T13:06:29.236 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-07T13:06:29.236 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-07T13:06:29.236 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-07T13:06:29.236 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-07T13:06:29.236 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-07T13:06:29.236 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-07T13:06:29.236 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-07T13:06:29.236 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:29.236 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-07T13:06:29.236 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:06:29.236 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-07T13:06:29.236 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-07T13:06:29.237 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:29.237 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-07T13:06:29.237 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-07T13:06:29.237 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-07T13:06:29.237 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-07T13:06:29.237 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-07T13:06:29.237 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-07T13:06:29.237 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-02-07T13:06:29.237 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:06:29.237 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T13:06:29.237 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:29.237 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-07T13:06:29.237 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:06:29.237 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-02-07T13:06:11.589778+0000", 2024-02-07T13:06:29.237 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T13:06:29.238 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:29.238 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-07T13:06:29.238 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-02-07T13:06:29.238 INFO:teuthology.orchestra.run.smithi067.stdout:mgr not available, waiting (3/15)... 2024-02-07T13:06:29.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:29 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/2219748145' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-07T13:06:29.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:29 smithi067 ceph-mon[25698]: mgrmap e3: y(active, since 1.00832s) 2024-02-07T13:06:31.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:31 smithi067 ceph-mon[25698]: mgrmap e4: y(active, since 2s) 2024-02-07T13:06:32.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:32 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/1429617226' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-07T13:06:32.562 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-02-07T13:06:32.562 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-02-07T13:06:32.562 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsid": "81df3e5a-c5b9-11ee-95b6-87774f69a715", 2024-02-07T13:06:32.562 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "health": { 2024-02-07T13:06:32.562 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-07T13:06:32.562 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-07T13:06:32.563 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-07T13:06:32.563 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:32.563 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-07T13:06:32.563 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-07T13:06:32.563 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 0 2024-02-07T13:06:32.563 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:06:32.563 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-07T13:06:32.563 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "a" 2024-02-07T13:06:32.563 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:06:32.564 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2024-02-07T13:06:32.564 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-07T13:06:32.564 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:06:32.564 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-07T13:06:32.564 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-07T13:06:32.564 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:32.564 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-07T13:06:32.564 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:06:32.564 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-07T13:06:32.565 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-07T13:06:32.565 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-07T13:06:32.565 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-07T13:06:32.565 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-07T13:06:32.565 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-07T13:06:32.570 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:32.570 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-07T13:06:32.571 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-07T13:06:32.571 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-07T13:06:32.571 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-07T13:06:32.571 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-07T13:06:32.571 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-07T13:06:32.571 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-07T13:06:32.571 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-07T13:06:32.571 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-07T13:06:32.571 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:32.571 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-07T13:06:32.571 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:06:32.571 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-07T13:06:32.571 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-07T13:06:32.571 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:32.571 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-07T13:06:32.572 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-07T13:06:32.572 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-07T13:06:32.572 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-07T13:06:32.572 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-07T13:06:32.572 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-07T13:06:32.572 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "restful" 2024-02-07T13:06:32.572 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ], 2024-02-07T13:06:32.572 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T13:06:32.572 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:32.572 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-07T13:06:32.572 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-07T13:06:32.572 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "modified": "2024-02-07T13:06:11.589778+0000", 2024-02-07T13:06:32.572 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-07T13:06:32.572 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout }, 2024-02-07T13:06:32.572 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-07T13:06:32.572 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-02-07T13:06:32.573 INFO:teuthology.orchestra.run.smithi067.stdout:mgr is available 2024-02-07T13:06:33.485 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:33 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/2075179355' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-07T13:06:33.486 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:33 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/2075179355' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-07T13:06:33.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-02-07T13:06:33.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [global] 2024-02-07T13:06:33.748 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout fsid = 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:06:33.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-07T13:06:33.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-07T13:06:33.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-07T13:06:33.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-07T13:06:33.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-02-07T13:06:33.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-07T13:06:33.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-07T13:06:33.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 2024-02-07T13:06:33.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout [osd] 2024-02-07T13:06:33.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-07T13:06:33.749 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-07T13:06:33.749 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling cephadm module... 2024-02-07T13:06:35.093 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:34 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/3838032277' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-07T13:06:35.094 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:34 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: ignoring --setuser ceph since I am not root 2024-02-07T13:06:35.094 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:34 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: ignoring --setgroup ceph since I am not root 2024-02-07T13:06:35.704 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:35 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:35.386+0000 7f2e80aeb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T13:06:35.705 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:35 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:35.482+0000 7f2e80aeb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T13:06:35.993 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:35 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/3838032277' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-07T13:06:35.993 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:35 smithi067 ceph-mon[25698]: mgrmap e5: y(active, since 6s) 2024-02-07T13:06:36.247 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:36 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:36.149+0000 7f2e80aeb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T13:06:36.247 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:36 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:36.245+0000 7f2e80aeb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T13:06:36.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:36 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:36.388+0000 7f2e80aeb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T13:06:36.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:36 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:36.486+0000 7f2e80aeb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T13:06:36.587 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-02-07T13:06:36.587 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-07T13:06:36.587 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-07T13:06:36.587 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-07T13:06:36.587 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-07T13:06:36.587 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-02-07T13:06:36.587 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for the mgr to restart... 2024-02-07T13:06:36.587 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr epoch 5... 2024-02-07T13:06:36.826 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:36 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/1260699965' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-07T13:06:36.827 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:36 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:36.571+0000 7f2e80aeb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T13:06:37.156 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:36 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:36.852+0000 7f2e80aeb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T13:06:37.156 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:36 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:36.965+0000 7f2e80aeb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T13:06:37.156 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:37 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:37.055+0000 7f2e80aeb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T13:06:37.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:37 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:37.153+0000 7f2e80aeb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T13:06:37.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:37 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:37.330+0000 7f2e80aeb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T13:06:38.786 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:38 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:38.315+0000 7f2e80aeb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T13:06:38.786 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:38 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:38.448+0000 7f2e80aeb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T13:06:38.786 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:38 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:38.783+0000 7f2e80aeb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T13:06:39.055 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:38 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:38.875+0000 7f2e80aeb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T13:06:39.551 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:39 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:39.074+0000 7f2e80aeb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:06:39.551 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:39 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:39.260+0000 7f2e80aeb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T13:06:39.806 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:39 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:39.548+0000 7f2e80aeb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:06:40.306 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:39 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:39.808+0000 7f2e80aeb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T13:06:40.306 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:40 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:40.049+0000 7f2e80aeb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T13:06:41.306 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:40 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:40.868+0000 7f2e80aeb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T13:06:41.768 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:41 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:41.448+0000 7f2e80aeb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T13:06:41.768 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:41 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:41.548+0000 7f2e80aeb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T13:06:41.768 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:41 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:06:41.767+0000 7f2e80aeb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T13:06:42.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:41 smithi067 ceph-mon[25698]: Active manager daemon y restarted 2024-02-07T13:06:42.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:41 smithi067 ceph-mon[25698]: Activating manager daemon y 2024-02-07T13:06:42.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:41 smithi067 ceph-mon[25698]: osdmap e2: 0 total, 0 up, 0 in 2024-02-07T13:06:42.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:41 smithi067 ceph-mon[25698]: mgrmap e6: y(active, starting, since 0.00404886s) 2024-02-07T13:06:42.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:41 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:06:42.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:41 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-07T13:06:42.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:41 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-07T13:06:42.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:41 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-07T13:06:42.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:41 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-07T13:06:42.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:41 smithi067 ceph-mon[25698]: Manager daemon y is now available 2024-02-07T13:06:42.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:41 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' 2024-02-07T13:06:42.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:41 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' 2024-02-07T13:06:42.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:41 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' 2024-02-07T13:06:42.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:41 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:06:43.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:42 smithi067 ceph-mon[25698]: Found migration_current of "None". Setting to last migration. 2024-02-07T13:06:43.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:42 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:06:43.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:42 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:06:43.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:42 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-07T13:06:43.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:42 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-07T13:06:43.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:42 smithi067 ceph-mon[25698]: mgrmap e7: y(active, since 1.00574s) 2024-02-07T13:06:43.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-02-07T13:06:43.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-07T13:06:43.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-07T13:06:43.359 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-02-07T13:06:43.359 INFO:teuthology.orchestra.run.smithi067.stdout:mgr epoch 5 is available 2024-02-07T13:06:43.359 INFO:teuthology.orchestra.run.smithi067.stdout:Setting orchestrator backend to cephadm... 2024-02-07T13:06:44.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:43 smithi067 ceph-mon[25698]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-07T13:06:44.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:43 smithi067 ceph-mon[25698]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-07T13:06:44.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:43 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' 2024-02-07T13:06:44.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:43 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' 2024-02-07T13:06:45.014 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:44 smithi067 ceph-mon[25698]: [07/Feb/2024:13:06:43] ENGINE Bus STARTING 2024-02-07T13:06:45.014 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:44 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' 2024-02-07T13:06:45.014 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:44 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:06:45.014 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:44 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:06:45.015 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:44 smithi067 ceph-mon[25698]: mgrmap e8: y(active, since 2s) 2024-02-07T13:06:45.908 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:45 smithi067 ceph-mon[25698]: [07/Feb/2024:13:06:43] ENGINE Serving on https://172.21.15.67:7150 2024-02-07T13:06:45.908 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:45 smithi067 ceph-mon[25698]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:06:45.908 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:45 smithi067 ceph-mon[25698]: [07/Feb/2024:13:06:44] ENGINE Serving on http://172.21.15.67:8765 2024-02-07T13:06:45.908 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:45 smithi067 ceph-mon[25698]: [07/Feb/2024:13:06:44] ENGINE Bus STARTED 2024-02-07T13:06:45.911 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-07T13:06:45.911 INFO:teuthology.orchestra.run.smithi067.stdout:Generating ssh key... 2024-02-07T13:06:47.155 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:46 smithi067 ceph-mon[25698]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:06:47.155 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:46 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: Generating public/private rsa key pair. 2024-02-07T13:06:47.156 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:46 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: Your identification has been saved in /tmp/tmp9ya0go2j/key. 2024-02-07T13:06:47.156 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:46 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: Your public key has been saved in /tmp/tmp9ya0go2j/key.pub. 2024-02-07T13:06:47.156 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:46 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: The key fingerprint is: 2024-02-07T13:06:47.156 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:46 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: SHA256:vOAGljtuDxQ3ip6T9dCMusTQmYQBgxlK8PAPsjZ2qRY ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:06:47.156 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:46 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: The key's randomart image is: 2024-02-07T13:06:47.156 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:46 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: +---[RSA 3072]----+ 2024-02-07T13:06:47.156 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:46 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: |@+ | 2024-02-07T13:06:47.156 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:46 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: |=B | 2024-02-07T13:06:47.156 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:46 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: |+ = . o | 2024-02-07T13:06:47.156 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:46 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: | = *.O o | 2024-02-07T13:06:47.156 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:46 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: |oE=o@ + S | 2024-02-07T13:06:47.156 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:46 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: |o+=B * . . | 2024-02-07T13:06:47.156 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:46 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: | oO + + . | 2024-02-07T13:06:47.157 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:46 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: |.. +.+ | 2024-02-07T13:06:47.157 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:46 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: | ..... | 2024-02-07T13:06:47.157 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:06:46 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: +----[SHA256]-----+ 2024-02-07T13:06:48.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:47 smithi067 ceph-mon[25698]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:06:48.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:47 smithi067 ceph-mon[25698]: Generating ssh key... 2024-02-07T13:06:48.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:47 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' 2024-02-07T13:06:48.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:47 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' 2024-02-07T13:06:48.714 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDb6yGugxBMo27bEoespmWlt02CBNjYEy88QkV6qt7cDYGD0I9Zm3GK88dufdNuPZbE+nPjg3Fgxbeuu+0iwjxNzO70KsspZ7sPEQoc+MEHWPmYVEZwNFfqSxG6gEVUhGhtYpizR74uQEnLm23ZdlMVaSvq+QtaDhP96aZByxgGU7YOXVA+03uTpLZapkV2nhBNDXxtaIbou+W6yeaCAUY25U0G12f/Nvifboic7mO5fuqzQ4UNojvFM/9Fpgqr1lAorTW9je50TpTkGfuYD0Yn0ULKxww1G5isdlLGOtI+zIeB8zZS22h0MJnmGuXQZU2C3xGHOk6Wz8b+6klryHsoKG2A2h5I/a2z8Qg4Huowg3xPY9u9Aas7R4iq4ZeCGvZdXINjEYwlBMPIj3yxQpE0Xw81+zDFBcnH71AkqdFAPQ/gNxyOkoqbTWz5HEMuaRcMhdqwyD3HVgKNZmV2RAENn42E94P03SrOF8ojqSZX2TXF5BkatDmVwbsMJSqweI8= ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:06:48.715 INFO:teuthology.orchestra.run.smithi067.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-07T13:06:48.715 INFO:teuthology.orchestra.run.smithi067.stdout:Adding key to root@localhost authorized_keys... 2024-02-07T13:06:48.715 INFO:teuthology.orchestra.run.smithi067.stdout:Adding host smithi067... 2024-02-07T13:06:50.027 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:49 smithi067 ceph-mon[25698]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:06:51.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:50 smithi067 ceph-mon[25698]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi067", "addr": "172.21.15.67", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:06:52.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:51 smithi067 ceph-mon[25698]: Deploying cephadm binary to smithi067 2024-02-07T13:06:54.488 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Added host 'smithi067' with addr '172.21.15.67' 2024-02-07T13:06:54.489 INFO:teuthology.orchestra.run.smithi067.stdout:Deploying unmanaged mon service... 2024-02-07T13:06:55.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:54 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' 2024-02-07T13:06:55.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:54 smithi067 ceph-mon[25698]: Added host smithi067 2024-02-07T13:06:55.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:54 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:06:55.872 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-07T13:06:55.872 INFO:teuthology.orchestra.run.smithi067.stdout:Deploying unmanaged mgr service... 2024-02-07T13:06:56.507 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:56 smithi067 ceph-mon[25698]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:06:56.507 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:56 smithi067 ceph-mon[25698]: Saving service mon spec with placement count:5 2024-02-07T13:06:56.507 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:56 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' 2024-02-07T13:06:58.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:57 smithi067 ceph-mon[25698]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:06:58.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:57 smithi067 ceph-mon[25698]: Saving service mgr spec with placement count:2 2024-02-07T13:06:58.031 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:57 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' 2024-02-07T13:06:58.032 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:57 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' 2024-02-07T13:06:58.034 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-07T13:06:59.636 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:59 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' 2024-02-07T13:06:59.636 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:06:59 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/2076305905' entity='client.admin' 2024-02-07T13:07:00.814 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling the dashboard module... 2024-02-07T13:07:01.465 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:01 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/417179147' entity='client.admin' 2024-02-07T13:07:02.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:02 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/2525297382' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-07T13:07:02.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:02 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' 2024-02-07T13:07:02.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:02 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' 2024-02-07T13:07:02.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:02 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:07:02.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:02 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' 2024-02-07T13:07:02.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:02 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi067", "caps": []}]: dispatch 2024-02-07T13:07:02.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:02 smithi067 ceph-mon[25698]: from='mgr.14120 172.21.15.67:0/3475358233' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi067", "caps": []}]': finished 2024-02-07T13:07:02.436 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:02 smithi067 ceph-mon[25698]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:02.436 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:02 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: ignoring --setuser ceph since I am not root 2024-02-07T13:07:02.437 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:02 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: ignoring --setgroup ceph since I am not root 2024-02-07T13:07:02.952 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:02 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:02.700+0000 7f43f87fd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T13:07:02.952 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:02 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:02.803+0000 7f43f87fd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T13:07:03.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:03 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/2525297382' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-07T13:07:03.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:03 smithi067 ceph-mon[25698]: mgrmap e9: y(active, since 20s) 2024-02-07T13:07:03.722 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:03 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:03.479+0000 7f43f87fd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T13:07:03.722 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:03 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:03.575+0000 7f43f87fd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T13:07:03.722 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:03 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:03.721+0000 7f43f87fd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T13:07:03.954 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-02-07T13:07:03.954 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-07T13:07:03.954 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-07T13:07:03.954 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-07T13:07:03.954 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-07T13:07:03.954 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-02-07T13:07:03.955 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for the mgr to restart... 2024-02-07T13:07:03.955 INFO:teuthology.orchestra.run.smithi067.stdout:Waiting for mgr epoch 9... 2024-02-07T13:07:04.056 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:03 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:03.819+0000 7f43f87fd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T13:07:04.056 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:03 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:03.904+0000 7f43f87fd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T13:07:04.401 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:04 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/849318205' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-07T13:07:04.401 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:04 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:04.180+0000 7f43f87fd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T13:07:04.402 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:04 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:04.308+0000 7f43f87fd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T13:07:04.402 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:04 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:04.400+0000 7f43f87fd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T13:07:04.679 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:04 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:04.499+0000 7f43f87fd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T13:07:05.056 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:04 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:04.677+0000 7f43f87fd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T13:07:06.056 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:05.669+0000 7f43f87fd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T13:07:06.056 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:05.803+0000 7f43f87fd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T13:07:06.441 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:06 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:06.143+0000 7f43f87fd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T13:07:06.441 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:06 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:06.236+0000 7f43f87fd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T13:07:06.441 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:06 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:06.439+0000 7f43f87fd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:07:06.806 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:06 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:06.631+0000 7f43f87fd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T13:07:07.187 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:06 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:06.924+0000 7f43f87fd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:07:07.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:07 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:07.185+0000 7f43f87fd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T13:07:07.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:07 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:07.429+0000 7f43f87fd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T13:07:08.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:08 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:08.257+0000 7f43f87fd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T13:07:09.165 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:08 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:08.841+0000 7f43f87fd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T13:07:09.166 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:08 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:08.943+0000 7f43f87fd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T13:07:09.166 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:07:09 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:07:09.164+0000 7f43f87fd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T13:07:10.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:09 smithi067 ceph-mon[25698]: Active manager daemon y restarted 2024-02-07T13:07:10.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:09 smithi067 ceph-mon[25698]: Activating manager daemon y 2024-02-07T13:07:10.180 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:09 smithi067 ceph-mon[25698]: osdmap e3: 0 total, 0 up, 0 in 2024-02-07T13:07:10.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:09 smithi067 ceph-mon[25698]: mgrmap e10: y(active, starting, since 0.00306179s) 2024-02-07T13:07:10.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:07:10.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-07T13:07:10.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-07T13:07:10.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-07T13:07:10.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-07T13:07:10.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:09 smithi067 ceph-mon[25698]: Manager daemon y is now available 2024-02-07T13:07:10.181 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:10.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:10.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:10.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-07T13:07:10.182 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-07T13:07:11.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:11 smithi067 ceph-mon[25698]: mgrmap e11: y(active, since 1.00582s) 2024-02-07T13:07:11.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:11 smithi067 ceph-mon[25698]: [07/Feb/2024:13:07:10] ENGINE Bus STARTING 2024-02-07T13:07:11.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:11 smithi067 ceph-mon[25698]: [07/Feb/2024:13:07:10] ENGINE Serving on https://172.21.15.67:7150 2024-02-07T13:07:11.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:11 smithi067 ceph-mon[25698]: [07/Feb/2024:13:07:10] ENGINE Serving on http://172.21.15.67:8765 2024-02-07T13:07:11.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:11 smithi067 ceph-mon[25698]: [07/Feb/2024:13:07:10] ENGINE Bus STARTED 2024-02-07T13:07:11.296 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout { 2024-02-07T13:07:11.296 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-07T13:07:11.296 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-07T13:07:11.296 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout } 2024-02-07T13:07:11.296 INFO:teuthology.orchestra.run.smithi067.stdout:mgr epoch 9 is available 2024-02-07T13:07:11.296 INFO:teuthology.orchestra.run.smithi067.stdout:Generating a dashboard self-signed certificate... 2024-02-07T13:07:12.741 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:12.741 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:07:12.741 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:12.741 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi067", "caps": []}]: dispatch 2024-02-07T13:07:12.741 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:12 smithi067 ceph-mon[25698]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:07:12.741 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:12.742 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:12.742 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:12 smithi067 ceph-mon[25698]: mgrmap e12: y(active, since 3s) 2024-02-07T13:07:12.744 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-07T13:07:12.744 INFO:teuthology.orchestra.run.smithi067.stdout:Creating initial admin user... 2024-02-07T13:07:13.703 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:13 smithi067 ceph-mon[25698]: Deploying daemon agent.smithi067 on smithi067 2024-02-07T13:07:14.275 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Ko6wVRyQm9fDPPJlIT8IZuh8C6xlLHn7ohtgcl08sCMBUaYaYZWFC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1707311233, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-07T13:07:14.276 INFO:teuthology.orchestra.run.smithi067.stdout:Fetching dashboard port number... 2024-02-07T13:07:14.783 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:14 smithi067 ceph-mon[25698]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:07:14.783 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:15.538 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stdout 8443 2024-02-07T13:07:15.547 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-07T13:07:15.547 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout disabled 2024-02-07T13:07:15.554 INFO:teuthology.orchestra.run.smithi067.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-07T13:07:15.554 INFO:teuthology.orchestra.run.smithi067.stdout:systemctl: stdout inactive 2024-02-07T13:07:15.554 INFO:teuthology.orchestra.run.smithi067.stdout:firewalld.service is not enabled 2024-02-07T13:07:15.555 INFO:teuthology.orchestra.run.smithi067.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-07T13:07:15.557 INFO:teuthology.orchestra.run.smithi067.stdout:Ceph Dashboard is now available at: 2024-02-07T13:07:15.557 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:07:15.557 INFO:teuthology.orchestra.run.smithi067.stdout: URL: https://smithi067.front.sepia.ceph.com:8443/ 2024-02-07T13:07:15.557 INFO:teuthology.orchestra.run.smithi067.stdout: User: admin 2024-02-07T13:07:15.557 INFO:teuthology.orchestra.run.smithi067.stdout: Password: 7zg3skxwmj 2024-02-07T13:07:15.557 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:07:15.557 INFO:teuthology.orchestra.run.smithi067.stdout:Saving cluster configuration to /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config directory 2024-02-07T13:07:15.557 INFO:teuthology.orchestra.run.smithi067.stdout:Enabling autotune for osd_memory_target 2024-02-07T13:07:15.930 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:15 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/1290977207' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-07T13:07:16.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:16 smithi067 ceph-mon[25698]: mgrmap e13: y(active, since 6s) 2024-02-07T13:07:18.209 INFO:teuthology.orchestra.run.smithi067.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-07T13:07:18.209 INFO:teuthology.orchestra.run.smithi067.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-07T13:07:18.209 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:07:18.209 INFO:teuthology.orchestra.run.smithi067.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-07T13:07:18.209 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:07:18.209 INFO:teuthology.orchestra.run.smithi067.stdout:Or, if you are only running a single cluster on this host: 2024-02-07T13:07:18.209 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:07:18.209 INFO:teuthology.orchestra.run.smithi067.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-07T13:07:18.209 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:07:18.209 INFO:teuthology.orchestra.run.smithi067.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-07T13:07:18.209 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:07:18.209 INFO:teuthology.orchestra.run.smithi067.stdout: ceph telemetry on 2024-02-07T13:07:18.210 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:07:18.210 INFO:teuthology.orchestra.run.smithi067.stdout:For more information see: 2024-02-07T13:07:18.210 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:07:18.210 INFO:teuthology.orchestra.run.smithi067.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-07T13:07:18.210 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:07:18.210 INFO:teuthology.orchestra.run.smithi067.stdout:Bootstrap complete. 2024-02-07T13:07:18.252 INFO:tasks.cephadm:Fetching config... 2024-02-07T13:07:18.252 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-07T13:07:18.252 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-07T13:07:18.268 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-07T13:07:18.268 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-07T13:07:18.268 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-07T13:07:18.322 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-07T13:07:18.322 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-07T13:07:18.322 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-07T13:07:18.487 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-07T13:07:18.487 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-07T13:07:18.487 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-07T13:07:18.542 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-07T13:07:18.542 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDb6yGugxBMo27bEoespmWlt02CBNjYEy88QkV6qt7cDYGD0I9Zm3GK88dufdNuPZbE+nPjg3Fgxbeuu+0iwjxNzO70KsspZ7sPEQoc+MEHWPmYVEZwNFfqSxG6gEVUhGhtYpizR74uQEnLm23ZdlMVaSvq+QtaDhP96aZByxgGU7YOXVA+03uTpLZapkV2nhBNDXxtaIbou+W6yeaCAUY25U0G12f/Nvifboic7mO5fuqzQ4UNojvFM/9Fpgqr1lAorTW9je50TpTkGfuYD0Yn0ULKxww1G5isdlLGOtI+zIeB8zZS22h0MJnmGuXQZU2C3xGHOk6Wz8b+6klryHsoKG2A2h5I/a2z8Qg4Huowg3xPY9u9Aas7R4iq4ZeCGvZdXINjEYwlBMPIj3yxQpE0Xw81+zDFBcnH71AkqdFAPQ/gNxyOkoqbTWz5HEMuaRcMhdqwyD3HVgKNZmV2RAENn42E94P03SrOF8ojqSZX2TXF5BkatDmVwbsMJSqweI8= ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-07T13:07:18.683 INFO:teuthology.orchestra.run.smithi067.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDb6yGugxBMo27bEoespmWlt02CBNjYEy88QkV6qt7cDYGD0I9Zm3GK88dufdNuPZbE+nPjg3Fgxbeuu+0iwjxNzO70KsspZ7sPEQoc+MEHWPmYVEZwNFfqSxG6gEVUhGhtYpizR74uQEnLm23ZdlMVaSvq+QtaDhP96aZByxgGU7YOXVA+03uTpLZapkV2nhBNDXxtaIbou+W6yeaCAUY25U0G12f/Nvifboic7mO5fuqzQ4UNojvFM/9Fpgqr1lAorTW9je50TpTkGfuYD0Yn0ULKxww1G5isdlLGOtI+zIeB8zZS22h0MJnmGuXQZU2C3xGHOk6Wz8b+6klryHsoKG2A2h5I/a2z8Qg4Huowg3xPY9u9Aas7R4iq4ZeCGvZdXINjEYwlBMPIj3yxQpE0Xw81+zDFBcnH71AkqdFAPQ/gNxyOkoqbTWz5HEMuaRcMhdqwyD3HVgKNZmV2RAENn42E94P03SrOF8ojqSZX2TXF5BkatDmVwbsMJSqweI8= ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:07:18.698 DEBUG:teuthology.orchestra.run.smithi159:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDb6yGugxBMo27bEoespmWlt02CBNjYEy88QkV6qt7cDYGD0I9Zm3GK88dufdNuPZbE+nPjg3Fgxbeuu+0iwjxNzO70KsspZ7sPEQoc+MEHWPmYVEZwNFfqSxG6gEVUhGhtYpizR74uQEnLm23ZdlMVaSvq+QtaDhP96aZByxgGU7YOXVA+03uTpLZapkV2nhBNDXxtaIbou+W6yeaCAUY25U0G12f/Nvifboic7mO5fuqzQ4UNojvFM/9Fpgqr1lAorTW9je50TpTkGfuYD0Yn0ULKxww1G5isdlLGOtI+zIeB8zZS22h0MJnmGuXQZU2C3xGHOk6Wz8b+6klryHsoKG2A2h5I/a2z8Qg4Huowg3xPY9u9Aas7R4iq4ZeCGvZdXINjEYwlBMPIj3yxQpE0Xw81+zDFBcnH71AkqdFAPQ/gNxyOkoqbTWz5HEMuaRcMhdqwyD3HVgKNZmV2RAENn42E94P03SrOF8ojqSZX2TXF5BkatDmVwbsMJSqweI8= ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-07T13:07:18.744 INFO:teuthology.orchestra.run.smithi159.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDb6yGugxBMo27bEoespmWlt02CBNjYEy88QkV6qt7cDYGD0I9Zm3GK88dufdNuPZbE+nPjg3Fgxbeuu+0iwjxNzO70KsspZ7sPEQoc+MEHWPmYVEZwNFfqSxG6gEVUhGhtYpizR74uQEnLm23ZdlMVaSvq+QtaDhP96aZByxgGU7YOXVA+03uTpLZapkV2nhBNDXxtaIbou+W6yeaCAUY25U0G12f/Nvifboic7mO5fuqzQ4UNojvFM/9Fpgqr1lAorTW9je50TpTkGfuYD0Yn0ULKxww1G5isdlLGOtI+zIeB8zZS22h0MJnmGuXQZU2C3xGHOk6Wz8b+6klryHsoKG2A2h5I/a2z8Qg4Huowg3xPY9u9Aas7R4iq4ZeCGvZdXINjEYwlBMPIj3yxQpE0Xw81+zDFBcnH71AkqdFAPQ/gNxyOkoqbTWz5HEMuaRcMhdqwyD3HVgKNZmV2RAENn42E94P03SrOF8ojqSZX2TXF5BkatDmVwbsMJSqweI8= ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:07:18.760 DEBUG:teuthology.orchestra.run.smithi191:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDb6yGugxBMo27bEoespmWlt02CBNjYEy88QkV6qt7cDYGD0I9Zm3GK88dufdNuPZbE+nPjg3Fgxbeuu+0iwjxNzO70KsspZ7sPEQoc+MEHWPmYVEZwNFfqSxG6gEVUhGhtYpizR74uQEnLm23ZdlMVaSvq+QtaDhP96aZByxgGU7YOXVA+03uTpLZapkV2nhBNDXxtaIbou+W6yeaCAUY25U0G12f/Nvifboic7mO5fuqzQ4UNojvFM/9Fpgqr1lAorTW9je50TpTkGfuYD0Yn0ULKxww1G5isdlLGOtI+zIeB8zZS22h0MJnmGuXQZU2C3xGHOk6Wz8b+6klryHsoKG2A2h5I/a2z8Qg4Huowg3xPY9u9Aas7R4iq4ZeCGvZdXINjEYwlBMPIj3yxQpE0Xw81+zDFBcnH71AkqdFAPQ/gNxyOkoqbTWz5HEMuaRcMhdqwyD3HVgKNZmV2RAENn42E94P03SrOF8ojqSZX2TXF5BkatDmVwbsMJSqweI8= ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-07T13:07:18.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:18 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/2122325857' entity='client.admin' 2024-02-07T13:07:18.806 INFO:teuthology.orchestra.run.smithi191.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDb6yGugxBMo27bEoespmWlt02CBNjYEy88QkV6qt7cDYGD0I9Zm3GK88dufdNuPZbE+nPjg3Fgxbeuu+0iwjxNzO70KsspZ7sPEQoc+MEHWPmYVEZwNFfqSxG6gEVUhGhtYpizR74uQEnLm23ZdlMVaSvq+QtaDhP96aZByxgGU7YOXVA+03uTpLZapkV2nhBNDXxtaIbou+W6yeaCAUY25U0G12f/Nvifboic7mO5fuqzQ4UNojvFM/9Fpgqr1lAorTW9je50TpTkGfuYD0Yn0ULKxww1G5isdlLGOtI+zIeB8zZS22h0MJnmGuXQZU2C3xGHOk6Wz8b+6klryHsoKG2A2h5I/a2z8Qg4Huowg3xPY9u9Aas7R4iq4ZeCGvZdXINjEYwlBMPIj3yxQpE0Xw81+zDFBcnH71AkqdFAPQ/gNxyOkoqbTWz5HEMuaRcMhdqwyD3HVgKNZmV2RAENn42E94P03SrOF8ojqSZX2TXF5BkatDmVwbsMJSqweI8= ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:07:18.821 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-07T13:07:19.067 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:07:20.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:20.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:20.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:20.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:20.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:20.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:20.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:20.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:20.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:20.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:20.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:20.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:20.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:21.530 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-07T13:07:21.531 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-07T13:07:22.168 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:21 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/3354495186' entity='client.admin' 2024-02-07T13:07:22.168 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:21 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:22.168 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:21 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:22.546 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:07:24.898 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:24 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:24.898 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:24 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:26.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:26 smithi067 ceph-mon[25698]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:07:26.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:26 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:26.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:26 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:26.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:26 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:07:26.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:26 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:07:26.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:26 smithi067 ceph-mon[25698]: Updating smithi067:/etc/ceph/ceph.conf 2024-02-07T13:07:26.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:26 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:26.526 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:26 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:26.552 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi159 2024-02-07T13:07:26.552 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-02-07T13:07:26.552 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.conf 2024-02-07T13:07:26.570 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-02-07T13:07:26.570 DEBUG:teuthology.orchestra.run.smithi159:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:07:26.628 INFO:tasks.cephadm:Adding host smithi159 to orchestrator... 2024-02-07T13:07:26.628 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph orch host add smithi159 2024-02-07T13:07:26.867 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:07:27.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:27 smithi067 ceph-mon[25698]: Updating smithi067:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:07:27.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:27 smithi067 ceph-mon[25698]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:07:27.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:27 smithi067 ceph-mon[25698]: Updating smithi067:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:07:27.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:27 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:27.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:27 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:30.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:29 smithi067 ceph-mon[25698]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi159", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:07:31.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:30 smithi067 ceph-mon[25698]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:31.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:30 smithi067 ceph-mon[25698]: Deploying cephadm binary to smithi159 2024-02-07T13:07:33.158 INFO:teuthology.orchestra.run.smithi067.stdout:Added host 'smithi159' with addr '172.21.15.159' 2024-02-07T13:07:33.159 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:32 smithi067 ceph-mon[25698]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:33.808 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph orch host ls --format=json 2024-02-07T13:07:34.049 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:07:34.447 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:34 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:34.447 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:34 smithi067 ceph-mon[25698]: Added host smithi159 2024-02-07T13:07:34.447 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:34 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:34.447 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:34 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:35.494 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:35 smithi067 ceph-mon[25698]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:35.494 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:35 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:35.494 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:35 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:35.494 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:35 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:07:35.494 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:35 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:07:35.494 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:35 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:07:35.495 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:35 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:35.495 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:35 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:35.495 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:35 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi159", "caps": []}]: dispatch 2024-02-07T13:07:35.495 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:35 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi159", "caps": []}]': finished 2024-02-07T13:07:35.949 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:07:35.949 INFO:teuthology.orchestra.run.smithi067.stdout:[{"addr": "172.21.15.67", "hostname": "smithi067", "labels": [], "status": ""}, {"addr": "172.21.15.159", "hostname": "smithi159", "labels": [], "status": ""}] 2024-02-07T13:07:36.273 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:36 smithi067 ceph-mon[25698]: Updating smithi159:/etc/ceph/ceph.conf 2024-02-07T13:07:36.273 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:36 smithi067 ceph-mon[25698]: Updating smithi159:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:07:36.273 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:36 smithi067 ceph-mon[25698]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:07:36.273 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:36 smithi067 ceph-mon[25698]: Updating smithi159:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:07:36.566 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi191 2024-02-07T13:07:36.567 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-07T13:07:36.567 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/etc/ceph/ceph.conf 2024-02-07T13:07:36.586 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-07T13:07:36.586 DEBUG:teuthology.orchestra.run.smithi191:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:07:36.643 INFO:tasks.cephadm:Adding host smithi191 to orchestrator... 2024-02-07T13:07:36.643 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph orch host add smithi191 2024-02-07T13:07:36.880 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:07:37.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:37 smithi067 ceph-mon[25698]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:37.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:37 smithi067 ceph-mon[25698]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:07:37.268 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:37 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:38.440 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:38 smithi067 ceph-mon[25698]: Deploying daemon agent.smithi159 on smithi159 2024-02-07T13:07:38.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:38.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:38.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:38.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:07:38.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:07:38.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:38.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:38.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:07:38.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:07:38.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:38.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:38.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:38.441 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:39.544 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:39 smithi067 ceph-mon[25698]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:40.459 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:40 smithi067 ceph-mon[25698]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi191", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:07:40.459 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:40 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:40.459 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:40 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:40.459 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:40 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:40.459 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:40 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:41.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:41 smithi067 ceph-mon[25698]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:41.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:41 smithi067 ceph-mon[25698]: Deploying cephadm binary to smithi191 2024-02-07T13:07:41.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:41 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:41.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:41 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:42 smithi067 ceph-mon[25698]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:43.175 INFO:teuthology.orchestra.run.smithi067.stdout:Added host 'smithi191' with addr '172.21.15.191' 2024-02-07T13:07:43.798 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph orch host ls --format=json 2024-02-07T13:07:44.043 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:07:44.458 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:44 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:44.459 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:44 smithi067 ceph-mon[25698]: Added host smithi191 2024-02-07T13:07:44.459 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:44 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:44.459 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:44 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:45.204 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:45 smithi067 ceph-mon[25698]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:45.204 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:45 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:45.204 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:45 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:45.205 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:45 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi191", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:07:45.205 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:45 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:07:45.205 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:45 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:07:45.205 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:45 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:45.205 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:45 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:45.205 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:45 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi191", "caps": []}]: dispatch 2024-02-07T13:07:45.205 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:45 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi191", "caps": []}]': finished 2024-02-07T13:07:45.954 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:07:45.954 INFO:teuthology.orchestra.run.smithi067.stdout:[{"addr": "172.21.15.67", "hostname": "smithi067", "labels": [], "status": ""}, {"addr": "172.21.15.159", "hostname": "smithi159", "labels": [], "status": ""}, {"addr": "172.21.15.191", "hostname": "smithi191", "labels": [], "status": ""}] 2024-02-07T13:07:46.277 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:46 smithi067 ceph-mon[25698]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-07T13:07:46.277 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:46 smithi067 ceph-mon[25698]: Updating smithi191:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:07:46.277 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:46 smithi067 ceph-mon[25698]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:07:46.277 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:46 smithi067 ceph-mon[25698]: Updating smithi191:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:07:46.578 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-07T13:07:46.579 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph osd crush tunables default 2024-02-07T13:07:46.815 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:07:47.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:47 smithi067 ceph-mon[25698]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:47.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:47 smithi067 ceph-mon[25698]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:07:47.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:47 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:47.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:47 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:47.255 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:47 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:48.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:48 smithi067 ceph-mon[25698]: Deploying daemon agent.smithi191 on smithi191 2024-02-07T13:07:48.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:48.219 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:07:48.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:07:48.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:48.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:48.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:07:48.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:07:48.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:48.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:48.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:48.220 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:49.185 INFO:teuthology.orchestra.run.smithi067.stderr:adjusted tunables profile to default 2024-02-07T13:07:49.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:49 smithi067 ceph-mon[25698]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:49.516 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:49 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/3557004973' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-07T13:07:49.784 INFO:tasks.cephadm:Adding mon.a on smithi067 2024-02-07T13:07:49.784 INFO:tasks.cephadm:Adding mon.b on smithi159 2024-02-07T13:07:49.784 INFO:tasks.cephadm:Adding mon.c on smithi191 2024-02-07T13:07:49.784 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph orch apply mon '3;smithi067:172.21.15.67=a;smithi159:172.21.15.159=b;smithi191:172.21.15.191=c' 2024-02-07T13:07:50.470 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:07:50.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:50 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/3557004973' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-07T13:07:50.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:50 smithi067 ceph-mon[25698]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T13:07:50.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:50 smithi067 ceph-mon[25698]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:50.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:51.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:51 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:51.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:51 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:52.234 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled mon update... 2024-02-07T13:07:52.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:52 smithi067 ceph-mon[25698]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:52.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:52 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:52.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:52 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:52.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:52 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:07:52.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:52 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:07:52.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:52 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:52.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:52 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:07:52.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:52 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:07:52.850 DEBUG:teuthology.orchestra.run.smithi159:mon.b> sudo journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mon.b.service 2024-02-07T13:07:52.853 DEBUG:teuthology.orchestra.run.smithi191:mon.c> sudo journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mon.c.service 2024-02-07T13:07:52.855 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-07T13:07:52.855 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-07T13:07:53.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:53 smithi067 ceph-mon[25698]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi159:172.21.15.159=b;smithi191:172.21.15.191=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:07:53.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:53 smithi067 ceph-mon[25698]: Saving service mon spec with placement smithi067:172.21.15.67=a;smithi159:172.21.15.159=b;smithi191:172.21.15.191=c;count:3 2024-02-07T13:07:53.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:53 smithi067 ceph-mon[25698]: Deploying daemon mon.c on smithi191 2024-02-07T13:07:53.885 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:07:55.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:54 smithi067 ceph-mon[25698]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:56.831 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-07T13:07:56.831 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":1,"fsid":"81df3e5a-c5b9-11ee-95b6-87774f69a715","modified":"2024-02-07T13:05:59.714918Z","created":"2024-02-07T13:05:59.714918Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-07T13:07:56.833 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 1 2024-02-07T13:07:56.913 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: mon.c@-1(???) e0 preinit fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:07:56.913 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: mon.c@-1(synchronizing).mds e1 new map 2024-02-07T13:07:56.913 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-07T13:07:56.913 INFO:journalctl@ceph.mon.c.smithi191.stdout: e1 2024-02-07T13:07:56.913 INFO:journalctl@ceph.mon.c.smithi191.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-07T13:07:56.913 INFO:journalctl@ceph.mon.c.smithi191.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-07T13:07:56.914 INFO:journalctl@ceph.mon.c.smithi191.stdout: legacy client fscid: -1 2024-02-07T13:07:56.914 INFO:journalctl@ceph.mon.c.smithi191.stdout: 2024-02-07T13:07:56.914 INFO:journalctl@ceph.mon.c.smithi191.stdout: No filesystems configured 2024-02-07T13:07:56.914 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-07T13:07:56.914 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-07T13:07:56.914 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-07T13:07:57.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:07:56 smithi067 ceph-mon[25698]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:57.183 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-07T13:07:57.183 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-07T13:07:57.183 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-07T13:07:57.184 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-07T13:07:57.184 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T13:07:57.184 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T13:07:57.184 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T13:07:57.184 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.184 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: Added host smithi159 2024-02-07T13:07:57.184 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:57.184 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.184 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:57.184 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.184 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.184 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:07:57.184 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:07:57.184 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:07:57.185 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.185 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.185 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi159", "caps": []}]: dispatch 2024-02-07T13:07:57.185 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi159", "caps": []}]': finished 2024-02-07T13:07:57.185 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: Updating smithi159:/etc/ceph/ceph.conf 2024-02-07T13:07:57.185 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: Updating smithi159:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:07:57.185 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:07:57.185 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: Updating smithi159:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:07:57.185 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:57.185 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:07:57.185 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.185 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: Deploying daemon agent.smithi159 on smithi159 2024-02-07T13:07:57.185 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.185 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.185 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:57.186 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:07:57.186 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:07:57.186 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.186 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:57.186 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:07:57.187 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:07:57.187 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.187 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.187 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.187 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.187 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:57.187 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi191", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:07:57.187 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.187 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.187 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.187 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.187 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:57.187 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: Deploying cephadm binary to smithi191 2024-02-07T13:07:57.187 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.187 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.188 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:57.188 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.188 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: Added host smithi191 2024-02-07T13:07:57.188 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:57.188 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.188 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:57.188 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.188 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.188 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi191", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:07:57.188 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:07:57.188 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:07:57.188 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.188 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.188 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi191", "caps": []}]: dispatch 2024-02-07T13:07:57.189 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi191", "caps": []}]': finished 2024-02-07T13:07:57.189 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-07T13:07:57.189 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: Updating smithi191:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:07:57.189 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:07:57.189 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: Updating smithi191:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:07:57.189 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:57.189 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:07:57.189 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.189 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.189 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.189 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: Deploying daemon agent.smithi191 on smithi191 2024-02-07T13:07:57.189 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:57.189 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:07:57.189 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:07:57.189 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.190 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:57.190 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:07:57.190 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:07:57.190 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.190 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.190 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.190 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.190 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:57.190 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/3557004973' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-07T13:07:57.190 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/3557004973' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-07T13:07:57.190 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T13:07:57.190 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:57.190 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.190 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.190 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.190 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:57.191 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.191 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:07:57.191 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:07:57.191 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:07:57.191 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:07:57.191 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:07:57.191 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:07:57.191 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi159:172.21.15.159=b;smithi191:172.21.15.191=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:07:57.191 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: Saving service mon spec with placement smithi067:172.21.15.67=a;smithi159:172.21.15.159=b;smithi191:172.21.15.191=c;count:3 2024-02-07T13:07:57.191 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: Deploying daemon mon.c on smithi191 2024-02-07T13:07:57.191 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:57.191 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:07:57.191 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:56 smithi191 ceph-mon[27623]: mon.c@-1(synchronizing).paxosservice(auth 1..6) refresh upgraded, format 0 -> 3 2024-02-07T13:07:57.191 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:57 smithi191 bash[27572]: f1e3ce748477ef97631d64f1d77c519a7b344b08dc256c2d36ba0c4b79ce8d36 2024-02-07T13:07:57.493 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:07:57 smithi191 systemd[1]: Started Ceph mon.c for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:07:58.520 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-07T13:07:58.521 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-07T13:07:58.767 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:08:02.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:08:02.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: mon.a calling monitor election 2024-02-07T13:08:02.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:02.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:02.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:02.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:02.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: mon.c calling monitor election 2024-02-07T13:08:02.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:02.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:02.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:02.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:02.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-07T13:08:02.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: monmap e2: 2 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T13:08:02.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: fsmap 2024-02-07T13:08:02.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T13:08:02.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: mgrmap e13: y(active, since 52s) 2024-02-07T13:08:02.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: overall HEALTH_OK 2024-02-07T13:08:02.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:02.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:02.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:02.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:02.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:02.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:02.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:02.308 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:02 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:02.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:08:02.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: mon.a calling monitor election 2024-02-07T13:08:02.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:02.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:02.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:02.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:02.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: mon.c calling monitor election 2024-02-07T13:08:02.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:02.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:02.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:02.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:02.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-07T13:08:02.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: monmap e2: 2 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T13:08:02.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: fsmap 2024-02-07T13:08:02.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T13:08:02.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: mgrmap e13: y(active, since 52s) 2024-02-07T13:08:02.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: overall HEALTH_OK 2024-02-07T13:08:02.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:02.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:02.371 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:02.371 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:02.371 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:02.371 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:02.371 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:02.371 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:02 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:03.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:03 smithi067 ceph-mon[25698]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:03.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:03 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:03.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:03 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:08:03.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:03 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:03.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:03 smithi067 ceph-mon[25698]: Deploying daemon mon.b on smithi159 2024-02-07T13:08:03.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:03 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:03.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:03 smithi191 ceph-mon[27623]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:03.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:03 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:03.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:03 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:08:03.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:03 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:03.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:03 smithi191 ceph-mon[27623]: Deploying daemon mon.b on smithi159 2024-02-07T13:08:03.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:03 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:03.479 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-07T13:08:03.480 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":2,"fsid":"81df3e5a-c5b9-11ee-95b6-87774f69a715","modified":"2024-02-07T13:07:56.919832Z","created":"2024-02-07T13:05:59.714918Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:3300","nonce":0},{"type":"v1","addr":"172.21.15.191:6789","nonce":0}]},"addr":"172.21.15.191:6789/0","public_addr":"172.21.15.191:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-07T13:08:03.481 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 2 2024-02-07T13:08:04.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:04 smithi067 ceph-mon[25698]: from='client.? 172.21.15.191:0/1294360931' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T13:08:04.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:04 smithi191 ceph-mon[27623]: from='client.? 172.21.15.191:0/1294360931' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T13:08:04.981 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-07T13:08:04.981 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph mon dump -f json 2024-02-07T13:08:05.225 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:08:05.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:05 smithi067 ceph-mon[25698]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:05.345 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:05 smithi191 ceph-mon[27623]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:05.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:05 smithi159 podman[27423]: 2024-02-07 13:08:05.646812135 +0000 UTC m=+0.018659019 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T13:08:06.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 podman[27423]: 2024-02-07 13:08:06.041210459 +0000 UTC m=+0.413057337 container init fe28fbc6f0a8e443f7378a579643174e4883d301c69ed5a6a46d4eab051f8cc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-02-07T13:08:06.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 podman[27423]: 2024-02-07 13:08:06.046967924 +0000 UTC m=+0.418814801 container start fe28fbc6f0a8e443f7378a579643174e4883d301c69ed5a6a46d4eab051f8cc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-b, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-07T13:08:06.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: set uid:gid to 167:167 (ceph:ceph) 2024-02-07T13:08:06.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: ceph version 19.0.0-1252-g44458cce (44458ccedb18241cc2e17f10cdab3d1385399261) squid (dev), process ceph-mon, pid 7 2024-02-07T13:08:06.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: pidfile_write: ignore empty --pid-file 2024-02-07T13:08:06.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: load: jerasure load: lrc 2024-02-07T13:08:06.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: RocksDB version: 7.9.2 2024-02-07T13:08:06.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Git sha 0 2024-02-07T13:08:06.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Compile date 2024-02-05 19:41:51 2024-02-07T13:08:06.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: DB SUMMARY 2024-02-07T13:08:06.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: DB Session ID: H5Z9PD1EFC93UCXM0K2M 2024-02-07T13:08:06.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: CURRENT file: CURRENT 2024-02-07T13:08:06.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: IDENTITY file: IDENTITY 2024-02-07T13:08:06.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-07T13:08:06.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-07T13:08:06.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-02-07T13:08:06.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.error_if_exists: 0 2024-02-07T13:08:06.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.create_if_missing: 0 2024-02-07T13:08:06.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.paranoid_checks: 1 2024-02-07T13:08:06.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-07T13:08:06.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-07T13:08:06.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-07T13:08:06.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.env: 0x561f448e4ae0 2024-02-07T13:08:06.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.fs: PosixFileSystem 2024-02-07T13:08:06.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.info_log: 0x561f469ab8c0 2024-02-07T13:08:06.445 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_file_opening_threads: 16 2024-02-07T13:08:06.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.statistics: (nil) 2024-02-07T13:08:06.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.use_fsync: 0 2024-02-07T13:08:06.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_log_file_size: 0 2024-02-07T13:08:06.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-07T13:08:06.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-07T13:08:06.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.keep_log_file_num: 1000 2024-02-07T13:08:06.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.recycle_log_file_num: 0 2024-02-07T13:08:06.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.allow_fallocate: 1 2024-02-07T13:08:06.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.allow_mmap_reads: 0 2024-02-07T13:08:06.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.allow_mmap_writes: 0 2024-02-07T13:08:06.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.use_direct_reads: 0 2024-02-07T13:08:06.447 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-07T13:08:06.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.create_missing_column_families: 0 2024-02-07T13:08:06.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.db_log_dir: 2024-02-07T13:08:06.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.wal_dir: 2024-02-07T13:08:06.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-07T13:08:06.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-07T13:08:06.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-07T13:08:06.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-07T13:08:06.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-07T13:08:06.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-07T13:08:06.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.advise_random_on_open: 1 2024-02-07T13:08:06.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.db_write_buffer_size: 0 2024-02-07T13:08:06.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.write_buffer_manager: 0x561f4697f400 2024-02-07T13:08:06.448 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-07T13:08:06.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-07T13:08:06.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-07T13:08:06.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.rate_limiter: (nil) 2024-02-07T13:08:06.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-07T13:08:06.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.wal_recovery_mode: 2 2024-02-07T13:08:06.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.enable_thread_tracking: 0 2024-02-07T13:08:06.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.enable_pipelined_write: 0 2024-02-07T13:08:06.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.unordered_write: 0 2024-02-07T13:08:06.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-07T13:08:06.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-07T13:08:06.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-07T13:08:06.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-07T13:08:06.449 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.row_cache: None 2024-02-07T13:08:06.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.wal_filter: None 2024-02-07T13:08:06.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-07T13:08:06.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.allow_ingest_behind: 0 2024-02-07T13:08:06.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.two_write_queues: 0 2024-02-07T13:08:06.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.manual_wal_flush: 0 2024-02-07T13:08:06.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.wal_compression: 0 2024-02-07T13:08:06.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.atomic_flush: 0 2024-02-07T13:08:06.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-07T13:08:06.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-07T13:08:06.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-07T13:08:06.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.log_readahead_size: 0 2024-02-07T13:08:06.450 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-07T13:08:06.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.best_efforts_recovery: 0 2024-02-07T13:08:06.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-07T13:08:06.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-07T13:08:06.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.allow_data_in_errors: 0 2024-02-07T13:08:06.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.db_host_id: __hostname__ 2024-02-07T13:08:06.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-07T13:08:06.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_background_jobs: 2 2024-02-07T13:08:06.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_background_compactions: -1 2024-02-07T13:08:06.451 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_subcompactions: 1 2024-02-07T13:08:06.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-07T13:08:06.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-07T13:08:06.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-07T13:08:06.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_total_wal_size: 0 2024-02-07T13:08:06.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-07T13:08:06.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-07T13:08:06.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-07T13:08:06.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-07T13:08:06.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_open_files: -1 2024-02-07T13:08:06.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.bytes_per_sync: 0 2024-02-07T13:08:06.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-07T13:08:06.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-07T13:08:06.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compaction_readahead_size: 0 2024-02-07T13:08:06.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_background_flushes: -1 2024-02-07T13:08:06.452 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Compression algorithms supported: 2024-02-07T13:08:06.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-07T13:08:06.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: kZSTD supported: 0 2024-02-07T13:08:06.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: kXpressCompression supported: 0 2024-02-07T13:08:06.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: kLZ4HCCompression supported: 1 2024-02-07T13:08:06.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: kZlibCompression supported: 1 2024-02-07T13:08:06.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: kSnappyCompression supported: 1 2024-02-07T13:08:06.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: kLZ4Compression supported: 1 2024-02-07T13:08:06.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: kBZip2Compression supported: 0 2024-02-07T13:08:06.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-07T13:08:06.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-07T13:08:06.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-02-07T13:08:06.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-07T13:08:06.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-07T13:08:06.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.merge_operator: 2024-02-07T13:08:06.453 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compaction_filter: None 2024-02-07T13:08:06.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compaction_filter_factory: None 2024-02-07T13:08:06.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.sst_partitioner_factory: None 2024-02-07T13:08:06.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-07T13:08:06.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-07T13:08:06.454 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561f469ab9c0) 2024-02-07T13:08:06.454 INFO:journalctl@ceph.mon.b.smithi159.stdout: cache_index_and_filter_blocks: 1 2024-02-07T13:08:06.454 INFO:journalctl@ceph.mon.b.smithi159.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-07T13:08:06.454 INFO:journalctl@ceph.mon.b.smithi159.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-07T13:08:06.454 INFO:journalctl@ceph.mon.b.smithi159.stdout: pin_top_level_index_and_filter: 1 2024-02-07T13:08:06.454 INFO:journalctl@ceph.mon.b.smithi159.stdout: index_type: 0 2024-02-07T13:08:06.454 INFO:journalctl@ceph.mon.b.smithi159.stdout: data_block_index_type: 0 2024-02-07T13:08:06.454 INFO:journalctl@ceph.mon.b.smithi159.stdout: index_shortening: 1 2024-02-07T13:08:06.454 INFO:journalctl@ceph.mon.b.smithi159.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-07T13:08:06.454 INFO:journalctl@ceph.mon.b.smithi159.stdout: checksum: 4 2024-02-07T13:08:06.454 INFO:journalctl@ceph.mon.b.smithi159.stdout: no_block_cache: 0 2024-02-07T13:08:06.454 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_cache: 0x561f469af090 2024-02-07T13:08:06.455 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_cache_name: BinnedLRUCache 2024-02-07T13:08:06.455 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_cache_options: 2024-02-07T13:08:06.455 INFO:journalctl@ceph.mon.b.smithi159.stdout: capacity : 536870912 2024-02-07T13:08:06.455 INFO:journalctl@ceph.mon.b.smithi159.stdout: num_shard_bits : 4 2024-02-07T13:08:06.455 INFO:journalctl@ceph.mon.b.smithi159.stdout: strict_capacity_limit : 0 2024-02-07T13:08:06.455 INFO:journalctl@ceph.mon.b.smithi159.stdout: high_pri_pool_ratio: 0.000 2024-02-07T13:08:06.455 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_cache_compressed: (nil) 2024-02-07T13:08:06.455 INFO:journalctl@ceph.mon.b.smithi159.stdout: persistent_cache: (nil) 2024-02-07T13:08:06.455 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_size: 4096 2024-02-07T13:08:06.455 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_size_deviation: 10 2024-02-07T13:08:06.455 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_restart_interval: 16 2024-02-07T13:08:06.455 INFO:journalctl@ceph.mon.b.smithi159.stdout: index_block_restart_interval: 1 2024-02-07T13:08:06.455 INFO:journalctl@ceph.mon.b.smithi159.stdout: metadata_block_size: 4096 2024-02-07T13:08:06.455 INFO:journalctl@ceph.mon.b.smithi159.stdout: partition_filters: 0 2024-02-07T13:08:06.455 INFO:journalctl@ceph.mon.b.smithi159.stdout: use_delta_encoding: 1 2024-02-07T13:08:06.455 INFO:journalctl@ceph.mon.b.smithi159.stdout: filter_policy: bloomfilter 2024-02-07T13:08:06.456 INFO:journalctl@ceph.mon.b.smithi159.stdout: whole_key_filtering: 1 2024-02-07T13:08:06.456 INFO:journalctl@ceph.mon.b.smithi159.stdout: verify_compression: 0 2024-02-07T13:08:06.456 INFO:journalctl@ceph.mon.b.smithi159.stdout: read_amp_bytes_per_bit: 0 2024-02-07T13:08:06.456 INFO:journalctl@ceph.mon.b.smithi159.stdout: format_version: 5 2024-02-07T13:08:06.456 INFO:journalctl@ceph.mon.b.smithi159.stdout: enable_index_compression: 1 2024-02-07T13:08:06.456 INFO:journalctl@ceph.mon.b.smithi159.stdout: block_align: 0 2024-02-07T13:08:06.456 INFO:journalctl@ceph.mon.b.smithi159.stdout: max_auto_readahead_size: 262144 2024-02-07T13:08:06.456 INFO:journalctl@ceph.mon.b.smithi159.stdout: prepopulate_block_cache: 0 2024-02-07T13:08:06.456 INFO:journalctl@ceph.mon.b.smithi159.stdout: initial_auto_readahead_size: 8192 2024-02-07T13:08:06.456 INFO:journalctl@ceph.mon.b.smithi159.stdout: num_file_reads_for_auto_readahead: 2 2024-02-07T13:08:06.456 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.write_buffer_size: 33554432 2024-02-07T13:08:06.456 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_write_buffer_number: 2 2024-02-07T13:08:06.456 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compression: NoCompression 2024-02-07T13:08:06.456 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.bottommost_compression: Disabled 2024-02-07T13:08:06.456 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.prefix_extractor: nullptr 2024-02-07T13:08:06.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-07T13:08:06.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.num_levels: 7 2024-02-07T13:08:06.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-07T13:08:06.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-07T13:08:06.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-07T13:08:06.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-07T13:08:06.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-07T13:08:06.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-07T13:08:06.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-07T13:08:06.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-07T13:08:06.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-07T13:08:06.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-07T13:08:06.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-07T13:08:06.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-07T13:08:06.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-07T13:08:06.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compression_opts.level: 32767 2024-02-07T13:08:06.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compression_opts.strategy: 0 2024-02-07T13:08:06.457 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-07T13:08:06.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-07T13:08:06.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-07T13:08:06.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-07T13:08:06.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compression_opts.enabled: false 2024-02-07T13:08:06.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-07T13:08:06.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-07T13:08:06.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-07T13:08:06.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-07T13:08:06.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.target_file_size_base: 67108864 2024-02-07T13:08:06.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-07T13:08:06.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-07T13:08:06.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-07T13:08:06.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-07T13:08:06.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-07T13:08:06.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-07T13:08:06.458 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-07T13:08:06.459 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-07T13:08:06.459 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-07T13:08:06.459 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-07T13:08:06.459 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-07T13:08:06.459 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-07T13:08:06.459 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-07T13:08:06.459 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-07T13:08:06.459 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.arena_block_size: 1048576 2024-02-07T13:08:06.459 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-07T13:08:06.459 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-07T13:08:06.459 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.disable_auto_compactions: 0 2024-02-07T13:08:06.459 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-07T13:08:06.459 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-07T13:08:06.459 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-07T13:08:06.459 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-07T13:08:06.459 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-07T13:08:06.459 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-07T13:08:06.460 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-07T13:08:06.460 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-07T13:08:06.460 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-07T13:08:06.460 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-07T13:08:06.460 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.table_properties_collectors: 2024-02-07T13:08:06.460 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.inplace_update_support: 0 2024-02-07T13:08:06.460 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-07T13:08:06.460 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-07T13:08:06.460 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-07T13:08:06.460 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-07T13:08:06.460 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.bloom_locality: 0 2024-02-07T13:08:06.460 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.max_successive_merges: 0 2024-02-07T13:08:06.460 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-07T13:08:06.460 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.paranoid_file_checks: 0 2024-02-07T13:08:06.461 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.force_consistency_checks: 1 2024-02-07T13:08:06.461 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.report_bg_io_stats: 0 2024-02-07T13:08:06.461 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.ttl: 2592000 2024-02-07T13:08:06.461 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-07T13:08:06.461 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-07T13:08:06.461 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-07T13:08:06.461 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.enable_blob_files: false 2024-02-07T13:08:06.461 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.min_blob_size: 0 2024-02-07T13:08:06.461 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.blob_file_size: 268435456 2024-02-07T13:08:06.461 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-07T13:08:06.461 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-07T13:08:06.461 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-07T13:08:06.461 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-07T13:08:06.461 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-07T13:08:06.461 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.blob_file_starting_level: 0 2024-02-07T13:08:06.462 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-07T13:08:06.462 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-07T13:08:06.462 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-07T13:08:06.462 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 73078abc-a7a3-4021-8fd4-9c1b44ae32d4 2024-02-07T13:08:06.462 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707311286076572, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-07T13:08:06.462 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-07T13:08:06.462 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707311286077108, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707311286, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "73078abc-a7a3-4021-8fd4-9c1b44ae32d4", "db_session_id": "H5Z9PD1EFC93UCXM0K2M", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-07T13:08:06.462 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707311286077208, "job": 1, "event": "recovery_finished"} 2024-02-07T13:08:06.462 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-07T13:08:06.462 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-07T13:08:06.462 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561f46aa0000 2024-02-07T13:08:06.462 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: DB pointer 0x561f46a8a000 2024-02-07T13:08:06.462 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-07T13:08:06.462 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-07T13:08:06.462 INFO:journalctl@ceph.mon.b.smithi159.stdout: ** DB Stats ** 2024-02-07T13:08:06.463 INFO:journalctl@ceph.mon.b.smithi159.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-07T13:08:06.463 INFO:journalctl@ceph.mon.b.smithi159.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-07T13:08:06.463 INFO:journalctl@ceph.mon.b.smithi159.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-07T13:08:06.463 INFO:journalctl@ceph.mon.b.smithi159.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-07T13:08:06.463 INFO:journalctl@ceph.mon.b.smithi159.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-07T13:08:06.463 INFO:journalctl@ceph.mon.b.smithi159.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-07T13:08:06.463 INFO:journalctl@ceph.mon.b.smithi159.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-07T13:08:06.463 INFO:journalctl@ceph.mon.b.smithi159.stdout: 2024-02-07T13:08:06.463 INFO:journalctl@ceph.mon.b.smithi159.stdout: ** Compaction Stats [default] ** 2024-02-07T13:08:06.463 INFO:journalctl@ceph.mon.b.smithi159.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-07T13:08:06.463 INFO:journalctl@ceph.mon.b.smithi159.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-07T13:08:06.463 INFO:journalctl@ceph.mon.b.smithi159.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-07T13:08:06.463 INFO:journalctl@ceph.mon.b.smithi159.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-07T13:08:06.463 INFO:journalctl@ceph.mon.b.smithi159.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-07T13:08:06.464 INFO:journalctl@ceph.mon.b.smithi159.stdout: 2024-02-07T13:08:06.464 INFO:journalctl@ceph.mon.b.smithi159.stdout: ** Compaction Stats [default] ** 2024-02-07T13:08:06.464 INFO:journalctl@ceph.mon.b.smithi159.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-07T13:08:06.464 INFO:journalctl@ceph.mon.b.smithi159.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-07T13:08:06.464 INFO:journalctl@ceph.mon.b.smithi159.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-07T13:08:06.464 INFO:journalctl@ceph.mon.b.smithi159.stdout: 2024-02-07T13:08:06.464 INFO:journalctl@ceph.mon.b.smithi159.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-07T13:08:06.464 INFO:journalctl@ceph.mon.b.smithi159.stdout: 2024-02-07T13:08:06.464 INFO:journalctl@ceph.mon.b.smithi159.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-07T13:08:06.464 INFO:journalctl@ceph.mon.b.smithi159.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-07T13:08:06.464 INFO:journalctl@ceph.mon.b.smithi159.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-07T13:08:06.464 INFO:journalctl@ceph.mon.b.smithi159.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-07T13:08:06.464 INFO:journalctl@ceph.mon.b.smithi159.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-07T13:08:06.464 INFO:journalctl@ceph.mon.b.smithi159.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-07T13:08:06.464 INFO:journalctl@ceph.mon.b.smithi159.stdout: Cumulative compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-07T13:08:06.466 INFO:journalctl@ceph.mon.b.smithi159.stdout: Interval compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-07T13:08:06.466 INFO:journalctl@ceph.mon.b.smithi159.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-07T13:08:06.466 INFO:journalctl@ceph.mon.b.smithi159.stdout: Block cache BinnedLRUCache@0x561f469af090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-07T13:08:06.466 INFO:journalctl@ceph.mon.b.smithi159.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-07T13:08:06.466 INFO:journalctl@ceph.mon.b.smithi159.stdout: 2024-02-07T13:08:06.466 INFO:journalctl@ceph.mon.b.smithi159.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-07T13:08:06.466 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-07T13:08:06.466 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: using public_addr v2:172.21.15.159:0/0 -> [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] 2024-02-07T13:08:06.466 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: starting mon.b rank -1 at public addrs [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] at bind addrs [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:08:06.466 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: mon.b@-1(???) e0 preinit fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:08:06.466 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: mon.b@-1(synchronizing).mds e1 new map 2024-02-07T13:08:06.466 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: mon.b@-1(synchronizing).mds e1 print_map 2024-02-07T13:08:06.466 INFO:journalctl@ceph.mon.b.smithi159.stdout: e1 2024-02-07T13:08:06.466 INFO:journalctl@ceph.mon.b.smithi159.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-07T13:08:06.467 INFO:journalctl@ceph.mon.b.smithi159.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-07T13:08:06.467 INFO:journalctl@ceph.mon.b.smithi159.stdout: legacy client fscid: -1 2024-02-07T13:08:06.467 INFO:journalctl@ceph.mon.b.smithi159.stdout: 2024-02-07T13:08:06.467 INFO:journalctl@ceph.mon.b.smithi159.stdout: No filesystems configured 2024-02-07T13:08:06.467 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-07T13:08:06.467 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-07T13:08:06.467 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-07T13:08:06.467 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-07T13:08:06.467 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-07T13:08:06.467 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-07T13:08:06.467 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-07T13:08:06.467 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T13:08:06.467 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T13:08:06.467 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-07T13:08:06.467 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: Added host smithi159 2024-02-07T13:08:06.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:06.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:06.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:08:06.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:06.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:06.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi159", "caps": []}]: dispatch 2024-02-07T13:08:06.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi159", "caps": []}]': finished 2024-02-07T13:08:06.468 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: Updating smithi159:/etc/ceph/ceph.conf 2024-02-07T13:08:06.469 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: Updating smithi159:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:08:06.469 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:08:06.469 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: Updating smithi159:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:08:06.469 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:06.469 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:08:06.469 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.469 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: Deploying daemon agent.smithi159 on smithi159 2024-02-07T13:08:06.469 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.469 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.469 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:06.469 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:06.469 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:06.469 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.469 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:06.469 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:06.469 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:06.470 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.470 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.470 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.470 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.470 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:06.470 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi191", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:08:06.470 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.470 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.470 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.470 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.470 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:06.470 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: Deploying cephadm binary to smithi191 2024-02-07T13:08:06.470 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.470 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.470 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:06.471 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.471 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: Added host smithi191 2024-02-07T13:08:06.471 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:06.471 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.471 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:06.471 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.471 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.471 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi191", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:08:06.471 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:06.471 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:06.471 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.471 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.471 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi191", "caps": []}]: dispatch 2024-02-07T13:08:06.471 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi191", "caps": []}]': finished 2024-02-07T13:08:06.471 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-07T13:08:06.472 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: Updating smithi191:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:08:06.472 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:08:06.472 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: Updating smithi191:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:08:06.472 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:06.472 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:08:06.472 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.472 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.472 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.472 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: Deploying daemon agent.smithi191 on smithi191 2024-02-07T13:08:06.472 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:06.472 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:06.472 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:06.472 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.472 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:06.472 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:06.473 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:06.473 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.473 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.473 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.473 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.473 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:06.473 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/3557004973' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-07T13:08:06.473 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/3557004973' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-07T13:08:06.473 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T13:08:06.473 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:06.473 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.473 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.473 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.473 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:06.473 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.473 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:06.474 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:06.474 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:06.474 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.474 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:08:06.474 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:06.474 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi067:172.21.15.67=a;smithi159:172.21.15.159=b;smithi191:172.21.15.191=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:08:06.474 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: Saving service mon spec with placement smithi067:172.21.15.67=a;smithi159:172.21.15.159=b;smithi191:172.21.15.191=c;count:3 2024-02-07T13:08:06.474 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: Deploying daemon mon.c on smithi191 2024-02-07T13:08:06.474 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:06.474 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:06.474 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:08:06.474 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: mon.a calling monitor election 2024-02-07T13:08:06.474 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:06.474 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: mon.c calling monitor election 2024-02-07T13:08:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-07T13:08:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: monmap e2: 2 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T13:08:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: fsmap 2024-02-07T13:08:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T13:08:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: mgrmap e13: y(active, since 52s) 2024-02-07T13:08:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: overall HEALTH_OK 2024-02-07T13:08:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.475 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:06.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:06.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:08:06.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:06.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: Deploying daemon mon.b on smithi159 2024-02-07T13:08:06.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:06.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: from='client.? 172.21.15.191:0/1294360931' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T13:08:06.476 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:06.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 ceph-mon[27461]: mon.b@-1(synchronizing).paxosservice(auth 1..6) refresh upgraded, format 0 -> 3 2024-02-07T13:08:06.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 bash[27423]: fe28fbc6f0a8e443f7378a579643174e4883d301c69ed5a6a46d4eab051f8cc5 2024-02-07T13:08:06.477 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:06 smithi159 systemd[1]: Started Ceph mon.b for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:08:11.326 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:11.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:08:11.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: mon.a calling monitor election 2024-02-07T13:08:11.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: mon.c calling monitor election 2024-02-07T13:08:11.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:11.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:11.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:11.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:11.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:11.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: mon.b calling monitor election 2024-02-07T13:08:11.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:11.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:11.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:11.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:11.328 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-07T13:08:11.328 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: monmap e3: 3 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T13:08:11.328 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: fsmap 2024-02-07T13:08:11.328 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T13:08:11.328 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: mgrmap e13: y(active, since 61s) 2024-02-07T13:08:11.328 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: overall HEALTH_OK 2024-02-07T13:08:11.328 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:11.328 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:11.328 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:11.328 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:11.328 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:11 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:11.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:11.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:08:11.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: mon.a calling monitor election 2024-02-07T13:08:11.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: mon.c calling monitor election 2024-02-07T13:08:11.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:11.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:11.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:11.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:11.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:11.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: mon.b calling monitor election 2024-02-07T13:08:11.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:11.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:11.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:11.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:11.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-07T13:08:11.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: monmap e3: 3 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T13:08:11.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: fsmap 2024-02-07T13:08:11.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T13:08:11.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: mgrmap e13: y(active, since 61s) 2024-02-07T13:08:11.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: overall HEALTH_OK 2024-02-07T13:08:11.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:11.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:11.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:11.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:11.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:11 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:11.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:11.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:08:11.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: mon.a calling monitor election 2024-02-07T13:08:11.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: mon.c calling monitor election 2024-02-07T13:08:11.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:11.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:08:11.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:11.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:11.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:11.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: mon.b calling monitor election 2024-02-07T13:08:11.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:11.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:11.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:11.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:11.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-07T13:08:11.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: monmap e3: 3 mons at {a=[v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0],b=[v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0],c=[v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-07T13:08:11.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: fsmap 2024-02-07T13:08:11.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: osdmap e4: 0 total, 0 up, 0 in 2024-02-07T13:08:11.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: mgrmap e13: y(active, since 61s) 2024-02-07T13:08:11.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: overall HEALTH_OK 2024-02-07T13:08:11.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:11.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:11.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:11.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:11.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:11 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.349 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:12.349 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.349 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.349 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:12.349 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:12.350 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:12.350 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: Updating smithi067:/etc/ceph/ceph.conf 2024-02-07T13:08:12.350 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: Updating smithi159:/etc/ceph/ceph.conf 2024-02-07T13:08:12.350 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-07T13:08:12.350 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: Updating smithi067:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:08:12.350 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: Updating smithi191:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:08:12.350 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: Updating smithi159:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:08:12.350 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.350 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.350 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.350 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.350 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.350 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.350 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.350 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: Metadata not up to date on all hosts. Skipping non agent specs 2024-02-07T13:08:12.351 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: Reconfiguring mon.a (unknown last config time)... 2024-02-07T13:08:12.351 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:08:12.351 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T13:08:12.351 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:12.351 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: Reconfiguring daemon mon.a on smithi067 2024-02-07T13:08:12.351 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:12 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:12.361 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:12.361 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.361 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.361 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:12.362 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:12.362 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:12.362 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: Updating smithi067:/etc/ceph/ceph.conf 2024-02-07T13:08:12.362 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: Updating smithi159:/etc/ceph/ceph.conf 2024-02-07T13:08:12.362 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-07T13:08:12.362 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: Updating smithi067:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:08:12.362 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: Updating smithi191:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:08:12.362 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: Updating smithi159:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:08:12.362 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.362 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.362 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.362 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.362 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.362 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.362 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.363 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: Metadata not up to date on all hosts. Skipping non agent specs 2024-02-07T13:08:12.363 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: Reconfiguring mon.a (unknown last config time)... 2024-02-07T13:08:12.363 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:08:12.363 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T13:08:12.363 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:12.363 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: Reconfiguring daemon mon.a on smithi067 2024-02-07T13:08:12.363 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:12 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:12.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:12.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:12.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:12.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:12.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: Updating smithi067:/etc/ceph/ceph.conf 2024-02-07T13:08:12.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: Updating smithi159:/etc/ceph/ceph.conf 2024-02-07T13:08:12.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-07T13:08:12.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: Updating smithi067:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:08:12.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: Updating smithi191:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:08:12.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: Updating smithi159:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:08:12.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:12.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: Metadata not up to date on all hosts. Skipping non agent specs 2024-02-07T13:08:12.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: Reconfiguring mon.a (unknown last config time)... 2024-02-07T13:08:12.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:08:12.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T13:08:12.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:12.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: Reconfiguring daemon mon.a on smithi067 2024-02-07T13:08:12.558 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:12 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:08:12.909 INFO:teuthology.orchestra.run.smithi191.stdout: 2024-02-07T13:08:12.909 INFO:teuthology.orchestra.run.smithi191.stdout:{"epoch":3,"fsid":"81df3e5a-c5b9-11ee-95b6-87774f69a715","modified":"2024-02-07T13:08:06.100804Z","created":"2024-02-07T13:05:59.714918Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:3300","nonce":0},{"type":"v1","addr":"172.21.15.191:6789","nonce":0}]},"addr":"172.21.15.191:6789/0","public_addr":"172.21.15.191:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:3300","nonce":0},{"type":"v1","addr":"172.21.15.159:6789","nonce":0}]},"addr":"172.21.15.159:6789/0","public_addr":"172.21.15.159:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-02-07T13:08:12.910 INFO:teuthology.orchestra.run.smithi191.stderr:dumped monmap epoch 3 2024-02-07T13:08:13.601 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:13 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:13.602 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:13 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:13.602 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:13 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:13.602 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:13 smithi067 ceph-mon[25698]: from='client.? 172.21.15.191:0/3153619661' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T13:08:13.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:13 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:13.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:13 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:13.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:13 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:13.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:13 smithi191 ceph-mon[27623]: from='client.? 172.21.15.191:0/3153619661' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T13:08:13.684 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-07T13:08:13.684 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph config generate-minimal-conf 2024-02-07T13:08:13.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:13 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:13.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:13 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:13.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:13 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:13.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:13 smithi159 ceph-mon[27461]: from='client.? 172.21.15.191:0/3153619661' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-07T13:08:13.925 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:08:14.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:14 smithi159 ceph-mon[27461]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:14.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:14.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:14.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:14 smithi159 ceph-mon[27461]: Reconfiguring mon.b (monmap changed)... 2024-02-07T13:08:14.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:08:14.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T13:08:14.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:14.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:14 smithi159 ceph-mon[27461]: Reconfiguring daemon mon.b on smithi159 2024-02-07T13:08:14.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:14.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:14.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:14.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:14.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:14.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:14.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:14 smithi067 ceph-mon[25698]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:15.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:14 smithi067 ceph-mon[25698]: Reconfiguring mon.b (monmap changed)... 2024-02-07T13:08:15.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:08:15.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T13:08:15.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:15.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:14 smithi067 ceph-mon[25698]: Reconfiguring daemon mon.b on smithi159 2024-02-07T13:08:15.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.057 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:14 smithi191 ceph-mon[27623]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:15.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:14 smithi191 ceph-mon[27623]: Reconfiguring mon.b (monmap changed)... 2024-02-07T13:08:15.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:08:15.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T13:08:15.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:15.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:14 smithi191 ceph-mon[27623]: Reconfiguring daemon mon.b on smithi159 2024-02-07T13:08:15.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.804 INFO:teuthology.orchestra.run.smithi067.stdout:# minimal ceph.conf for 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:08:15.804 INFO:teuthology.orchestra.run.smithi067.stdout:[global] 2024-02-07T13:08:15.804 INFO:teuthology.orchestra.run.smithi067.stdout: fsid = 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:08:15.804 INFO:teuthology.orchestra.run.smithi067.stdout: mon_host = [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] [v2:172.21.15.159:3300/0,v1:172.21.15.159:6789/0] [v2:172.21.15.191:3300/0,v1:172.21.15.191:6789/0] 2024-02-07T13:08:15.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:15 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:15 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:15.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:15 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:16.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:15 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:16.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:15 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:16.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:15 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:16.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:15 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:16.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:15 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:16.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:15 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:16.423 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-07T13:08:16.423 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-07T13:08:16.423 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.conf 2024-02-07T13:08:16.452 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-07T13:08:16.453 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:08:16.519 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-02-07T13:08:16.519 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.conf 2024-02-07T13:08:16.574 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-02-07T13:08:16.574 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:08:16.644 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-07T13:08:16.644 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.conf 2024-02-07T13:08:16.674 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-07T13:08:16.674 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:08:16.742 INFO:tasks.cephadm:Adding mgr.y on smithi067 2024-02-07T13:08:16.742 INFO:tasks.cephadm:Adding mgr.x on smithi159 2024-02-07T13:08:16.743 INFO:tasks.cephadm:Adding mgr.z on smithi191 2024-02-07T13:08:16.743 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph orch apply mgr '3;smithi067=y;smithi159=x;smithi191=z' 2024-02-07T13:08:16.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:16 smithi159 ceph-mon[27461]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:16.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:16 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:16.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:16 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:16.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:16 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/1084384031' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:16.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:16 smithi159 ceph-mon[27461]: Reconfiguring mon.c (monmap changed)... 2024-02-07T13:08:16.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:16 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:08:16.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:16 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T13:08:16.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:16 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:16.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:16 smithi159 ceph-mon[27461]: Reconfiguring daemon mon.c on smithi191 2024-02-07T13:08:17.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:16 smithi067 ceph-mon[25698]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:17.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:16 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:17.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:16 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:17.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:16 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/1084384031' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:17.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:16 smithi067 ceph-mon[25698]: Reconfiguring mon.c (monmap changed)... 2024-02-07T13:08:17.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:16 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:08:17.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:16 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T13:08:17.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:16 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:17.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:16 smithi067 ceph-mon[25698]: Reconfiguring daemon mon.c on smithi191 2024-02-07T13:08:17.057 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:16 smithi191 ceph-mon[27623]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:17.057 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:16 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:17.057 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:16 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:17.057 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:16 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/1084384031' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:17.057 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:16 smithi191 ceph-mon[27623]: Reconfiguring mon.c (monmap changed)... 2024-02-07T13:08:17.057 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:16 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-07T13:08:17.057 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:16 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-07T13:08:17.057 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:16 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:17.057 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:16 smithi191 ceph-mon[27623]: Reconfiguring daemon mon.c on smithi191 2024-02-07T13:08:17.420 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:08:18.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:18 smithi191 ceph-mon[27623]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:18.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:18 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:18.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:18 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:18.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:18 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:18.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:18 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:18.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:18 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:18.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:18 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:18 smithi067 ceph-mon[25698]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:18 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:18 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:18 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:18 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:18 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:18 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:18.649 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:18 smithi159 ceph-mon[27461]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:18.649 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:18 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:18.649 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:18 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:18.649 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:18 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:18.649 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:18 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:18.649 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:18 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:18.649 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:18 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:19.094 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled mgr update... 2024-02-07T13:08:19.656 DEBUG:teuthology.orchestra.run.smithi159:mgr.x> sudo journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mgr.x.service 2024-02-07T13:08:19.659 DEBUG:teuthology.orchestra.run.smithi191:mgr.z> sudo journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mgr.z.service 2024-02-07T13:08:19.661 INFO:tasks.cephadm:Deploying OSDs... 2024-02-07T13:08:19.661 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-07T13:08:19.661 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T13:08:19.679 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-07T13:08:19.679 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/nvme1n1 2024-02-07T13:08:19.736 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/nvme1n1 2024-02-07T13:08:19.736 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:08:19.736 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 764 Links: 1 Device type: 103,2 2024-02-07T13:08:19.736 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:08:19.736 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:08:19.736 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-02-07 13:08:18.834683706 +0000 2024-02-07T13:08:19.736 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-02-07 13:04:59.362278752 +0000 2024-02-07T13:08:19.736 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-02-07 13:04:59.362278752 +0000 2024-02-07T13:08:19.736 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-02-07 13:04:59.354278935 +0000 2024-02-07T13:08:19.737 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-07T13:08:19.805 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-02-07T13:08:19.805 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-02-07T13:08:19.805 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000283887 s, 1.8 MB/s 2024-02-07T13:08:19.806 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-07T13:08:19.862 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/nvme2n1 2024-02-07T13:08:19.918 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/nvme2n1 2024-02-07T13:08:19.918 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:08:19.918 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 774 Links: 1 Device type: 103,4 2024-02-07T13:08:19.918 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:08:19.918 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:08:19.918 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-02-07 13:08:18.845683455 +0000 2024-02-07T13:08:19.919 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-02-07 13:04:59.514271141 +0000 2024-02-07T13:08:19.919 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-02-07 13:04:59.514271141 +0000 2024-02-07T13:08:19.919 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-02-07 13:04:59.502271840 +0000 2024-02-07T13:08:19.919 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-07T13:08:19.985 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-02-07T13:08:19.985 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-02-07T13:08:19.985 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000240663 s, 2.1 MB/s 2024-02-07T13:08:19.986 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-07T13:08:20.042 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/nvme3n1 2024-02-07T13:08:20.098 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/nvme3n1 2024-02-07T13:08:20.098 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:08:20.098 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 787 Links: 1 Device type: 103,6 2024-02-07T13:08:20.098 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:08:20.098 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:08:20.098 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-02-07 13:08:18.856683204 +0000 2024-02-07T13:08:20.098 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-02-07 13:04:59.654262983 +0000 2024-02-07T13:08:20.098 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-02-07 13:04:59.654262983 +0000 2024-02-07T13:08:20.098 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-02-07 13:04:59.643263624 +0000 2024-02-07T13:08:20.099 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-07T13:08:20.165 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-02-07T13:08:20.165 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-02-07T13:08:20.165 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000206141 s, 2.5 MB/s 2024-02-07T13:08:20.166 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-07T13:08:20.221 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/nvme4n1 2024-02-07T13:08:20.276 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/nvme4n1 2024-02-07T13:08:20.276 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:08:20.276 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 799 Links: 1 Device type: 103,8 2024-02-07T13:08:20.277 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:08:20.277 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:08:20.277 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-02-07 13:08:18.866682976 +0000 2024-02-07T13:08:20.277 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-02-07 13:04:59.798254593 +0000 2024-02-07T13:08:20.277 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-02-07 13:04:59.798254593 +0000 2024-02-07T13:08:20.277 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-02-07 13:04:59.786255292 +0000 2024-02-07T13:08:20.277 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-07T13:08:20.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:20 smithi191 ceph-mon[27623]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi067=y;smithi159=x;smithi191=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:08:20.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:20 smithi191 ceph-mon[27623]: Saving service mgr spec with placement smithi067=y;smithi159=x;smithi191=z;count:3 2024-02-07T13:08:20.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:20 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:20.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:20 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:20.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:20 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:20.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:20 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:20.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:20 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:20.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:20 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T13:08:20.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:20 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-07T13:08:20.328 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:20 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T13:08:20.328 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:20 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:20.328 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:20 smithi191 ceph-mon[27623]: Deploying daemon mgr.z on smithi191 2024-02-07T13:08:20.342 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-02-07T13:08:20.342 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-02-07T13:08:20.342 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000198965 s, 2.6 MB/s 2024-02-07T13:08:20.343 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-07T13:08:20.399 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-02-07T13:08:20.399 DEBUG:teuthology.orchestra.run.smithi159:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T13:08:20.418 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-07T13:08:20.418 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/nvme1n1 2024-02-07T13:08:20.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:20 smithi159 ceph-mon[27461]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi067=y;smithi159=x;smithi191=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:08:20.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:20 smithi159 ceph-mon[27461]: Saving service mgr spec with placement smithi067=y;smithi159=x;smithi191=z;count:3 2024-02-07T13:08:20.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:20 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:20.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:20 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:20.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:20 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:20.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:20 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:20.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:20 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:20.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:20 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T13:08:20.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:20 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-07T13:08:20.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:20 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T13:08:20.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:20 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:20.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:20 smithi159 ceph-mon[27461]: Deploying daemon mgr.z on smithi191 2024-02-07T13:08:20.460 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/nvme1n1 2024-02-07T13:08:20.461 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:08:20.461 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 762 Links: 1 Device type: 103,2 2024-02-07T13:08:20.461 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:08:20.461 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:08:20.461 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-02-07 13:08:13.863796020 +0000 2024-02-07T13:08:20.461 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-02-07 13:05:00.980449468 +0000 2024-02-07T13:08:20.461 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-02-07 13:05:00.980449468 +0000 2024-02-07T13:08:20.462 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-02-07 13:05:00.969449675 +0000 2024-02-07T13:08:20.462 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-07T13:08:20.529 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-02-07T13:08:20.529 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-02-07T13:08:20.530 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000256085 s, 2.0 MB/s 2024-02-07T13:08:20.531 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-07T13:08:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:20 smithi067 ceph-mon[25698]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi067=y;smithi159=x;smithi191=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:08:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:20 smithi067 ceph-mon[25698]: Saving service mgr spec with placement smithi067=y;smithi159=x;smithi191=z;count:3 2024-02-07T13:08:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:20 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:20 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:20 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:20 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:20 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:20 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T13:08:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:20 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-07T13:08:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:20 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T13:08:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:20 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:20.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:20 smithi067 ceph-mon[25698]: Deploying daemon mgr.z on smithi191 2024-02-07T13:08:20.588 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/nvme2n1 2024-02-07T13:08:20.644 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/nvme2n1 2024-02-07T13:08:20.645 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:08:20.645 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 774 Links: 1 Device type: 103,4 2024-02-07T13:08:20.645 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:08:20.645 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:08:20.645 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-02-07 13:08:13.874795811 +0000 2024-02-07T13:08:20.645 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-02-07 13:05:01.133446573 +0000 2024-02-07T13:08:20.645 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-02-07 13:05:01.133446573 +0000 2024-02-07T13:08:20.645 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-02-07 13:05:01.120446819 +0000 2024-02-07T13:08:20.645 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-07T13:08:20.711 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-02-07T13:08:20.712 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-02-07T13:08:20.712 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000225753 s, 2.3 MB/s 2024-02-07T13:08:20.713 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-07T13:08:20.769 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/nvme3n1 2024-02-07T13:08:20.825 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/nvme3n1 2024-02-07T13:08:20.825 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:08:20.825 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 786 Links: 1 Device type: 103,6 2024-02-07T13:08:20.825 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:08:20.825 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:08:20.825 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-02-07 13:08:13.886795584 +0000 2024-02-07T13:08:20.826 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-02-07 13:05:01.281443773 +0000 2024-02-07T13:08:20.826 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-02-07 13:05:01.281443773 +0000 2024-02-07T13:08:20.826 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-02-07 13:05:01.268444018 +0000 2024-02-07T13:08:20.826 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-07T13:08:20.891 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-02-07T13:08:20.892 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-02-07T13:08:20.892 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000215256 s, 2.4 MB/s 2024-02-07T13:08:20.893 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-07T13:08:20.948 DEBUG:teuthology.orchestra.run.smithi159:> stat /dev/nvme4n1 2024-02-07T13:08:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: File: /dev/nvme4n1 2024-02-07T13:08:21.003 INFO:teuthology.orchestra.run.smithi159.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:08:21.003 INFO:teuthology.orchestra.run.smithi159.stdout:Device: 5h/5d Inode: 798 Links: 1 Device type: 103,8 2024-02-07T13:08:21.004 INFO:teuthology.orchestra.run.smithi159.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:08:21.004 INFO:teuthology.orchestra.run.smithi159.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:08:21.004 INFO:teuthology.orchestra.run.smithi159.stdout:Access: 2024-02-07 13:08:13.897795375 +0000 2024-02-07T13:08:21.004 INFO:teuthology.orchestra.run.smithi159.stdout:Modify: 2024-02-07 13:05:01.429440972 +0000 2024-02-07T13:08:21.004 INFO:teuthology.orchestra.run.smithi159.stdout:Change: 2024-02-07 13:05:01.429440972 +0000 2024-02-07T13:08:21.004 INFO:teuthology.orchestra.run.smithi159.stdout: Birth: 2024-02-07 13:05:01.417441199 +0000 2024-02-07T13:08:21.004 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-07T13:08:21.069 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records in 2024-02-07T13:08:21.069 INFO:teuthology.orchestra.run.smithi159.stderr:1+0 records out 2024-02-07T13:08:21.069 INFO:teuthology.orchestra.run.smithi159.stderr:512 bytes copied, 0.000231987 s, 2.2 MB/s 2024-02-07T13:08:21.070 DEBUG:teuthology.orchestra.run.smithi159:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-07T13:08:21.126 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-07T13:08:21.126 DEBUG:teuthology.orchestra.run.smithi191:> dd if=/scratch_devs of=/dev/stdout 2024-02-07T13:08:21.142 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-07T13:08:21.142 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/nvme1n1 2024-02-07T13:08:21.198 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/nvme1n1 2024-02-07T13:08:21.198 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:08:21.198 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 763 Links: 1 Device type: 103,2 2024-02-07T13:08:21.198 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:08:21.199 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:08:21.199 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-07 13:08:13.495926711 +0000 2024-02-07T13:08:21.199 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-07 13:05:02.628143790 +0000 2024-02-07T13:08:21.199 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-07 13:05:02.628143790 +0000 2024-02-07T13:08:21.199 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-02-07 13:05:02.619143944 +0000 2024-02-07T13:08:21.199 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-07T13:08:21.270 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-07T13:08:21.271 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-07T13:08:21.271 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000216176 s, 2.4 MB/s 2024-02-07T13:08:21.272 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-07T13:08:21.329 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/nvme2n1 2024-02-07T13:08:21.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:21 smithi191 ceph-mon[27623]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:21.382 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/nvme2n1 2024-02-07T13:08:21.383 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:08:21.383 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 775 Links: 1 Device type: 103,4 2024-02-07T13:08:21.383 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:08:21.383 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:08:21.383 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-07 13:08:13.506926529 +0000 2024-02-07T13:08:21.383 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-07 13:05:02.781141169 +0000 2024-02-07T13:08:21.383 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-07 13:05:02.781141169 +0000 2024-02-07T13:08:21.383 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-02-07 13:05:02.773141306 +0000 2024-02-07T13:08:21.383 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-07T13:08:21.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:21 smithi159 ceph-mon[27461]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:21.449 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-07T13:08:21.449 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-07T13:08:21.449 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000210119 s, 2.4 MB/s 2024-02-07T13:08:21.450 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-07T13:08:21.506 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/nvme3n1 2024-02-07T13:08:21.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:21 smithi067 ceph-mon[25698]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:21.562 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/nvme3n1 2024-02-07T13:08:21.562 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:08:21.562 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 786 Links: 1 Device type: 103,6 2024-02-07T13:08:21.562 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:08:21.562 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:08:21.562 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-07 13:08:13.517926346 +0000 2024-02-07T13:08:21.562 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-07 13:05:02.930138617 +0000 2024-02-07T13:08:21.562 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-07 13:05:02.930138617 +0000 2024-02-07T13:08:21.562 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-02-07 13:05:02.917138840 +0000 2024-02-07T13:08:21.563 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-07T13:08:21.629 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-07T13:08:21.629 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-07T13:08:21.630 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000212614 s, 2.4 MB/s 2024-02-07T13:08:21.631 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-07T13:08:21.687 DEBUG:teuthology.orchestra.run.smithi191:> stat /dev/nvme4n1 2024-02-07T13:08:21.700 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:21 smithi191 podman[29994]: 2024-02-07 13:08:21.675196065 +0000 UTC m=+0.345405590 container start b66654ac8d280fc9286acde1be31f61ea2bb73edca66cc541ff3be082b8a9f64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-02-07T13:08:21.714 INFO:teuthology.orchestra.run.smithi191.stdout: File: /dev/nvme4n1 2024-02-07T13:08:21.714 INFO:teuthology.orchestra.run.smithi191.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-07T13:08:21.714 INFO:teuthology.orchestra.run.smithi191.stdout:Device: 5h/5d Inode: 799 Links: 1 Device type: 103,8 2024-02-07T13:08:21.714 INFO:teuthology.orchestra.run.smithi191.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-07T13:08:21.715 INFO:teuthology.orchestra.run.smithi191.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-07T13:08:21.715 INFO:teuthology.orchestra.run.smithi191.stdout:Access: 2024-02-07 13:08:13.528926163 +0000 2024-02-07T13:08:21.715 INFO:teuthology.orchestra.run.smithi191.stdout:Modify: 2024-02-07 13:05:03.074136150 +0000 2024-02-07T13:08:21.715 INFO:teuthology.orchestra.run.smithi191.stdout:Change: 2024-02-07 13:05:03.074136150 +0000 2024-02-07T13:08:21.715 INFO:teuthology.orchestra.run.smithi191.stdout: Birth: 2024-02-07 13:05:03.062136356 +0000 2024-02-07T13:08:21.715 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-07T13:08:21.782 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records in 2024-02-07T13:08:21.782 INFO:teuthology.orchestra.run.smithi191.stderr:1+0 records out 2024-02-07T13:08:21.782 INFO:teuthology.orchestra.run.smithi191.stderr:512 bytes copied, 0.000206902 s, 2.5 MB/s 2024-02-07T13:08:21.784 DEBUG:teuthology.orchestra.run.smithi191:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-07T13:08:21.839 INFO:tasks.cephadm:Deploying osd.0 on smithi067 with /dev/nvme4n1... 2024-02-07T13:08:21.839 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-07T13:08:22.092 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:08:22.119 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:21 smithi191 ceph-mgr[30125]: -- 172.21.15.191:0/428378696 <== mon.1 v2:172.21.15.191:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b7c290d520 con 0x55b7c28e2c00 2024-02-07T13:08:22.119 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:21 smithi191 bash[29994]: b66654ac8d280fc9286acde1be31f61ea2bb73edca66cc541ff3be082b8a9f64 2024-02-07T13:08:22.119 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:21 smithi191 systemd[1]: Started Ceph mgr.z for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:08:22.619 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:22 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:22.220+0000 7fdc179a3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T13:08:22.619 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:22 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:22.311+0000 7fdc179a3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T13:08:23.119 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:22 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:22.945+0000 7fdc179a3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T13:08:23.119 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:23 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:23.036+0000 7fdc179a3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T13:08:23.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:22 smithi191 ceph-mon[27623]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:23.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:22 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:23.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:22 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:23.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:22 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:23.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:22 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T13:08:23.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:22 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-07T13:08:23.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:22 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T13:08:23.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:22 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:23.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:22 smithi191 ceph-mon[27623]: Deploying daemon mgr.x on smithi159 2024-02-07T13:08:23.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:22 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:23.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:22 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:23.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:22 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:23.160 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:22 smithi159 ceph-mon[27461]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:23.161 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:22 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:23.161 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:22 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:23.161 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:22 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:23.161 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:22 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T13:08:23.161 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:22 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-07T13:08:23.161 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:22 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T13:08:23.161 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:22 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:23.161 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:22 smithi159 ceph-mon[27461]: Deploying daemon mgr.x on smithi159 2024-02-07T13:08:23.161 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:22 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:23.161 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:22 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:23.161 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:22 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:23.225 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:22 smithi067 ceph-mon[25698]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:23.225 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:22 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:23.225 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:22 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:23.225 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:22 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:23.225 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:22 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T13:08:23.225 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:22 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-07T13:08:23.225 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:22 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T13:08:23.225 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:22 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:23.226 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:22 smithi067 ceph-mon[25698]: Deploying daemon mgr.x on smithi159 2024-02-07T13:08:23.226 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:22 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:23.226 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:22 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:23.226 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:22 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:23.611 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:23 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:23.176+0000 7fdc179a3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T13:08:23.611 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:23 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:23.270+0000 7fdc179a3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T13:08:23.611 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:23 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:23.350+0000 7fdc179a3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T13:08:23.611 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:23 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:23.610+0000 7fdc179a3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T13:08:23.872 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:23 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:23.717+0000 7fdc179a3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T13:08:23.872 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:23 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:23.801+0000 7fdc179a3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T13:08:23.943 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:23 smithi159 systemd[1]: Starting Ceph mgr.x for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:08:24.155 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping: /dev/nvme4n1 2024-02-07T13:08:24.155 INFO:teuthology.orchestra.run.smithi067.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-07T13:08:24.155 INFO:teuthology.orchestra.run.smithi067.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-07T13:08:24.156 INFO:teuthology.orchestra.run.smithi067.stderr: stderr: 10+0 records in 2024-02-07T13:08:24.156 INFO:teuthology.orchestra.run.smithi067.stderr:10+0 records out 2024-02-07T13:08:24.156 INFO:teuthology.orchestra.run.smithi067.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0142947 s, 734 MB/s 2024-02-07T13:08:24.156 INFO:teuthology.orchestra.run.smithi067.stderr:--> Zapping successful for: 2024-02-07T13:08:24.368 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:23 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:23.895+0000 7fdc179a3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T13:08:24.369 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:24 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:24.071+0000 7fdc179a3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T13:08:24.398 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:24 smithi159 podman[29135]: 2024-02-07 13:08:24.070882193 +0000 UTC m=+0.104726374 container create 86d280d6c1bf01d10da5f5e46f102818f3015620f546856a16f9df9d37e75dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x, ceph=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-07T13:08:24.398 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:24 smithi159 podman[29135]: 2024-02-07 13:08:23.984901052 +0000 UTC m=+0.018745187 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T13:08:24.399 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:24 smithi159 podman[29135]: 2024-02-07 13:08:24.289859034 +0000 UTC m=+0.323703169 container init 86d280d6c1bf01d10da5f5e46f102818f3015620f546856a16f9df9d37e75dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131) 2024-02-07T13:08:24.399 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:24 smithi159 podman[29135]: 2024-02-07 13:08:24.29593484 +0000 UTC m=+0.329778979 container start 86d280d6c1bf01d10da5f5e46f102818f3015620f546856a16f9df9d37e75dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131) 2024-02-07T13:08:24.399 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:24 smithi159 ceph-mgr[29175]: -- 172.21.15.159:0/3742968544 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557c74307520 con 0x557c742dcc00 2024-02-07T13:08:24.399 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:24 smithi159 bash[29135]: 86d280d6c1bf01d10da5f5e46f102818f3015620f546856a16f9df9d37e75dbc 2024-02-07T13:08:24.695 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:24 smithi159 systemd[1]: Started Ceph mgr.x for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:08:24.842 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi067:/dev/nvme4n1 2024-02-07T13:08:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:08:25.119 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:25 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:25.039+0000 7fdc179a3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T13:08:25.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:24 smithi191 ceph-mon[27623]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:25.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:24 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:25.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:24 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:25.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:24 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:25.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:24 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:25.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:24 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:25.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:24 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:25.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:24 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:25.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:24 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:25.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:24 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T13:08:25.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:24 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T13:08:25.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:24 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:25.193 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:24 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:24.877+0000 7fa9adb74200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T13:08:25.193 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:24 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:24.973+0000 7fa9adb74200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T13:08:25.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:24 smithi159 ceph-mon[27461]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:25.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:24 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:25.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:24 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:25.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:24 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:25.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:24 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:25.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:24 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:25.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:24 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:25.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:24 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:25.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:24 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:25.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:24 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T13:08:25.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:24 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T13:08:25.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:24 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:25.236 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:24 smithi067 ceph-mon[25698]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:25.236 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:24 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:25.236 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:24 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:25.236 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:24 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:25.236 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:24 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:25.237 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:24 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:25.237 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:24 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:25.237 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:24 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:25.237 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:24 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:25.237 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:24 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-07T13:08:25.237 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:24 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-07T13:08:25.237 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:24 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:25.505 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:25 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:25.167+0000 7fdc179a3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T13:08:25.505 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:25 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:25.504+0000 7fdc179a3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T13:08:25.785 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:25 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:25.592+0000 7fdc179a3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T13:08:25.785 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:25 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:25.784+0000 7fdc179a3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:08:25.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:25 smithi159 ceph-mon[27461]: Metadata not up to date on all hosts. Skipping non agent specs 2024-02-07T13:08:25.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:25 smithi159 ceph-mon[27461]: Reconfiguring mgr.y (unknown last config time)... 2024-02-07T13:08:25.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:25 smithi159 ceph-mon[27461]: Reconfiguring daemon mgr.y on smithi067 2024-02-07T13:08:25.944 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:25 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:25.663+0000 7fa9adb74200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T13:08:25.944 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:25 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:25.761+0000 7fa9adb74200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T13:08:25.944 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:25 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:25.906+0000 7fa9adb74200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T13:08:26.081 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:25 smithi067 ceph-mon[25698]: Metadata not up to date on all hosts. Skipping non agent specs 2024-02-07T13:08:26.081 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:25 smithi067 ceph-mon[25698]: Reconfiguring mgr.y (unknown last config time)... 2024-02-07T13:08:26.081 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:25 smithi067 ceph-mon[25698]: Reconfiguring daemon mgr.y on smithi067 2024-02-07T13:08:26.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:25 smithi191 ceph-mon[27623]: Metadata not up to date on all hosts. Skipping non agent specs 2024-02-07T13:08:26.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:25 smithi191 ceph-mon[27623]: Reconfiguring mgr.y (unknown last config time)... 2024-02-07T13:08:26.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:25 smithi191 ceph-mon[27623]: Reconfiguring daemon mgr.y on smithi067 2024-02-07T13:08:26.119 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:25 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:25.967+0000 7fdc179a3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T13:08:26.368 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:26 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:26.004+0000 7fa9adb74200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T13:08:26.369 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:26 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:26.100+0000 7fa9adb74200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T13:08:26.369 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:26 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:26.367+0000 7fa9adb74200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T13:08:26.514 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:26 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:26.246+0000 7fdc179a3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:08:26.514 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:26 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:26.513+0000 7fdc179a3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T13:08:26.673 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:26 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:26.511+0000 7fa9adb74200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T13:08:26.673 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:26 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:26.613+0000 7fa9adb74200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T13:08:26.834 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:26 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:26.756+0000 7fdc179a3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T13:08:26.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:26 smithi159 ceph-mon[27461]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:26.943 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:26 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:26.725+0000 7fa9adb74200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T13:08:26.943 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:26 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:26.915+0000 7fa9adb74200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T13:08:27.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:26 smithi067 ceph-mon[25698]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:27.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:26 smithi191 ceph-mon[27623]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:27.729 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:27 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:27.584+0000 7fdc179a3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T13:08:28.442 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:27 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:27.967+0000 7fa9adb74200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T13:08:28.442 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:28.099+0000 7fa9adb74200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T13:08:28.442 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:28.441+0000 7fa9adb74200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T13:08:28.484 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:28.219+0000 7fdc179a3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T13:08:28.484 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:28.328+0000 7fdc179a3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T13:08:28.693 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:28.531+0000 7fa9adb74200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T13:08:28.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-mon[27461]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:28.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:28.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:28.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:28.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-mon[27461]: Metadata not up to date on all hosts. Skipping non agent specs 2024-02-07T13:08:28.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-mon[27461]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:08:28.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T13:08:28.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T13:08:28.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:28.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.695 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.776 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:28 smithi067 ceph-mon[25698]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:28.776 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:28 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.776 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:28 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.776 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:28 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:28.776 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:28 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:28.777 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:28 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:28.777 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:28 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.777 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:28 smithi067 ceph-mon[25698]: Metadata not up to date on all hosts. Skipping non agent specs 2024-02-07T13:08:28.777 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:28 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.777 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:28 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.777 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:28 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.777 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:28 smithi067 ceph-mon[25698]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:08:28.777 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:28 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T13:08:28.777 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:28 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T13:08:28.777 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:28 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:28.777 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:28 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.777 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:28 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.777 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:28 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.816 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:08:28.562+0000 7fdc179a3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T13:08:28.816 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-mon[27623]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:28.816 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.816 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.817 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:28.817 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:28.817 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:28.817 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.817 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-mon[27623]: Metadata not up to date on all hosts. Skipping non agent specs 2024-02-07T13:08:28.817 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.817 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.817 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.817 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-mon[27623]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:08:28.817 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T13:08:28.817 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T13:08:28.817 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:28.817 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.817 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:28.817 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:28 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:29.193 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:28.725+0000 7fa9adb74200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:08:29.193 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:28 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:28.909+0000 7fa9adb74200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T13:08:29.443 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:29 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:29.192+0000 7fa9adb74200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:08:29.775 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:29 smithi067 ceph-mon[25698]: Standby manager daemon z started 2024-02-07T13:08:29.775 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:29 smithi067 ceph-mon[25698]: from='mgr.? 172.21.15.191:0/4242152392' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-07T13:08:29.775 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:29 smithi067 ceph-mon[25698]: from='mgr.? 172.21.15.191:0/4242152392' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:08:29.775 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:29 smithi067 ceph-mon[25698]: from='mgr.? 172.21.15.191:0/4242152392' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-07T13:08:29.775 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:29 smithi067 ceph-mon[25698]: from='mgr.? 172.21.15.191:0/4242152392' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:08:29.775 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:29 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:29.775 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:29 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:29.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:29 smithi191 ceph-mon[27623]: Standby manager daemon z started 2024-02-07T13:08:29.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:29 smithi191 ceph-mon[27623]: from='mgr.? 172.21.15.191:0/4242152392' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-07T13:08:29.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:29 smithi191 ceph-mon[27623]: from='mgr.? 172.21.15.191:0/4242152392' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:08:29.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:29 smithi191 ceph-mon[27623]: from='mgr.? 172.21.15.191:0/4242152392' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-07T13:08:29.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:29 smithi191 ceph-mon[27623]: from='mgr.? 172.21.15.191:0/4242152392' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:08:29.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:29 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:29.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:29 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:29.943 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:29 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:29.455+0000 7fa9adb74200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T13:08:29.943 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:29 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:29.691+0000 7fa9adb74200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T13:08:29.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:29 smithi159 ceph-mon[27461]: Standby manager daemon z started 2024-02-07T13:08:29.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:29 smithi159 ceph-mon[27461]: from='mgr.? 172.21.15.191:0/4242152392' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-07T13:08:29.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:29 smithi159 ceph-mon[27461]: from='mgr.? 172.21.15.191:0/4242152392' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:08:29.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:29 smithi159 ceph-mon[27461]: from='mgr.? 172.21.15.191:0/4242152392' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-07T13:08:29.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:29 smithi159 ceph-mon[27461]: from='mgr.? 172.21.15.191:0/4242152392' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:08:29.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:29 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:29.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:29 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:30.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:30 smithi067 ceph-mon[25698]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:30.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:30 smithi067 ceph-mon[25698]: mgrmap e14: y(active, since 80s), standbys: z 2024-02-07T13:08:30.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:30 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-07T13:08:30.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:30 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:30.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:30 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:30.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:30 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:30.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:30 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:30.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:30 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:30.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:30 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:30.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:30 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:30.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:30 smithi191 ceph-mon[27623]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:30.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:30 smithi191 ceph-mon[27623]: mgrmap e14: y(active, since 80s), standbys: z 2024-02-07T13:08:30.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:30 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-07T13:08:30.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:30 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:30.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:30 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:30.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:30 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:30.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:30 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:30.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:30 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:30.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:30 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:30.870 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:30 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:30.943 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:30 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:30.494+0000 7fa9adb74200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T13:08:30.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:30 smithi159 ceph-mon[27461]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:30.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:30 smithi159 ceph-mon[27461]: mgrmap e14: y(active, since 80s), standbys: z 2024-02-07T13:08:30.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:30 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-07T13:08:30.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:30 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:30.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:30 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:30.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:30 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:30.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:30 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:30.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:30 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:30.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:30 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:30.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:30 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:31.381 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:31 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:31.066+0000 7fa9adb74200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T13:08:31.381 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:31 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:31.165+0000 7fa9adb74200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T13:08:31.381 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:08:31 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:08:31.379+0000 7fa9adb74200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T13:08:31.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:31 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/630058009' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e53b131-f387-4600-b27c-e421ff58b822"}]: dispatch 2024-02-07T13:08:31.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:31 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/630058009' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e53b131-f387-4600-b27c-e421ff58b822"}]': finished 2024-02-07T13:08:31.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:31 smithi191 ceph-mon[27623]: osdmap e5: 1 total, 0 up, 1 in 2024-02-07T13:08:31.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:31 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:08:31.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:31 smithi191 ceph-mon[27623]: Standby manager daemon x started 2024-02-07T13:08:31.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:31 smithi191 ceph-mon[27623]: from='mgr.? 172.21.15.159:0/2311808869' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-07T13:08:31.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:31 smithi191 ceph-mon[27623]: from='mgr.? 172.21.15.159:0/2311808869' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:08:31.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:31 smithi191 ceph-mon[27623]: from='mgr.? 172.21.15.159:0/2311808869' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-07T13:08:31.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:31 smithi191 ceph-mon[27623]: from='mgr.? 172.21.15.159:0/2311808869' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:08:31.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:31 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/630058009' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e53b131-f387-4600-b27c-e421ff58b822"}]: dispatch 2024-02-07T13:08:31.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:31 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/630058009' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e53b131-f387-4600-b27c-e421ff58b822"}]': finished 2024-02-07T13:08:31.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:31 smithi159 ceph-mon[27461]: osdmap e5: 1 total, 0 up, 1 in 2024-02-07T13:08:31.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:31 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:08:31.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:31 smithi159 ceph-mon[27461]: Standby manager daemon x started 2024-02-07T13:08:31.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:31 smithi159 ceph-mon[27461]: from='mgr.? 172.21.15.159:0/2311808869' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-07T13:08:31.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:31 smithi159 ceph-mon[27461]: from='mgr.? 172.21.15.159:0/2311808869' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:08:31.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:31 smithi159 ceph-mon[27461]: from='mgr.? 172.21.15.159:0/2311808869' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-07T13:08:31.694 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:31 smithi159 ceph-mon[27461]: from='mgr.? 172.21.15.159:0/2311808869' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:08:31.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:31 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/630058009' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e53b131-f387-4600-b27c-e421ff58b822"}]: dispatch 2024-02-07T13:08:31.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:31 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/630058009' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e53b131-f387-4600-b27c-e421ff58b822"}]': finished 2024-02-07T13:08:31.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:31 smithi067 ceph-mon[25698]: osdmap e5: 1 total, 0 up, 1 in 2024-02-07T13:08:31.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:31 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:08:31.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:31 smithi067 ceph-mon[25698]: Standby manager daemon x started 2024-02-07T13:08:31.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:31 smithi067 ceph-mon[25698]: from='mgr.? 172.21.15.159:0/2311808869' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-07T13:08:31.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:31 smithi067 ceph-mon[25698]: from='mgr.? 172.21.15.159:0/2311808869' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:08:31.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:31 smithi067 ceph-mon[25698]: from='mgr.? 172.21.15.159:0/2311808869' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-07T13:08:31.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:31 smithi067 ceph-mon[25698]: from='mgr.? 172.21.15.159:0/2311808869' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:08:32.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:32 smithi159 ceph-mon[27461]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:32.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:32 smithi159 ceph-mon[27461]: mgrmap e15: y(active, since 82s), standbys: z, x 2024-02-07T13:08:32.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:32 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-07T13:08:32.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:32 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/2285350889' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T13:08:33.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:32 smithi067 ceph-mon[25698]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:33.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:32 smithi067 ceph-mon[25698]: mgrmap e15: y(active, since 82s), standbys: z, x 2024-02-07T13:08:33.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:32 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-07T13:08:33.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:32 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/2285350889' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T13:08:33.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:32 smithi191 ceph-mon[27623]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:33.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:32 smithi191 ceph-mon[27623]: mgrmap e15: y(active, since 82s), standbys: z, x 2024-02-07T13:08:33.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:32 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-07T13:08:33.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:32 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/2285350889' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T13:08:35.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:34 smithi067 ceph-mon[25698]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:35.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:34 smithi191 ceph-mon[27623]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:35.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:34 smithi159 ceph-mon[27461]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:37.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:36 smithi067 ceph-mon[25698]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:37.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:36 smithi191 ceph-mon[27623]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:37.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:36 smithi159 ceph-mon[27461]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:39.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:38 smithi191 ceph-mon[27623]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:39.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:38 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-07T13:08:39.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:38 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:39.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:38 smithi191 ceph-mon[27623]: Deploying daemon osd.0 on smithi067 2024-02-07T13:08:39.131 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:38 smithi067 ceph-mon[25698]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:39.131 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-07T13:08:39.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:39.132 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:38 smithi067 ceph-mon[25698]: Deploying daemon osd.0 on smithi067 2024-02-07T13:08:39.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:38 smithi159 ceph-mon[27461]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:39.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:38 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-07T13:08:39.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:38 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:39.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:38 smithi159 ceph-mon[27461]: Deploying daemon osd.0 on smithi067 2024-02-07T13:08:41.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:40 smithi067 ceph-mon[25698]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:41.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:40 smithi191 ceph-mon[27623]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:41.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:40 smithi159 ceph-mon[27461]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:43.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:42 smithi191 ceph-mon[27623]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:43.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:42 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:43.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:42 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:43.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:42 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:43.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:42 smithi159 ceph-mon[27461]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:43.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:42 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:43.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:42 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:43.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:42 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:42 smithi067 ceph-mon[25698]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:42 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:42 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:42 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:45.052 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:44 smithi067 ceph-mon[25698]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:45.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:44 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:45.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:44 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:45.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:44 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:45.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:44 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:45.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:44 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:45.053 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:44 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:45.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:44 smithi191 ceph-mon[27623]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:45.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:44 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:45.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:44 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:45.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:44 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:45.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:44 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:45.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:44 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:45.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:44 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:45.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:44 smithi159 ceph-mon[27461]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:45.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:44 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:45.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:44 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:45.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:44 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:45.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:44 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:45.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:44 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:45.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:44 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:47.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:46 smithi067 ceph-mon[25698]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:47.118 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:46 smithi191 ceph-mon[27623]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:47.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:46 smithi159 ceph-mon[27461]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:48.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:47 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:48.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:47 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:48.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:47 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:48.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:47 smithi067 ceph-mon[25698]: from='osd.0 [v2:172.21.15.67:6802/4131814377,v1:172.21.15.67:6803/4131814377]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-07T13:08:48.141 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 0 on host 'smithi067' 2024-02-07T13:08:48.221 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:47 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:48.221 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:47 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:48.221 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:47 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:48.221 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:47 smithi191 ceph-mon[27623]: from='osd.0 [v2:172.21.15.67:6802/4131814377,v1:172.21.15.67:6803/4131814377]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-07T13:08:48.432 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:47 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:48.432 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:47 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:48.433 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:47 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:48.433 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:47 smithi159 ceph-mon[27461]: from='osd.0 [v2:172.21.15.67:6802/4131814377,v1:172.21.15.67:6803/4131814377]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-07T13:08:48.726 DEBUG:teuthology.orchestra.run.smithi067:osd.0> sudo journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@osd.0.service 2024-02-07T13:08:48.728 INFO:tasks.cephadm:Deploying osd.1 on smithi159 with /dev/nvme4n1... 2024-02-07T13:08:48.729 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-07T13:08:48.968 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.b/config 2024-02-07T13:08:49.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:48 smithi191 ceph-mon[27623]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:49.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:48 smithi191 ceph-mon[27623]: from='osd.0 [v2:172.21.15.67:6802/4131814377,v1:172.21.15.67:6803/4131814377]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-07T13:08:49.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:48 smithi191 ceph-mon[27623]: osdmap e6: 1 total, 0 up, 1 in 2024-02-07T13:08:49.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:48 smithi191 ceph-mon[27623]: from='osd.0 [v2:172.21.15.67:6802/4131814377,v1:172.21.15.67:6803/4131814377]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-07T13:08:49.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:48 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:08:49.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:48 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:48 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:48 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:49.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:48 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:08:49.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:48 smithi191 ceph-mon[27623]: Adjusting osd_memory_target on smithi067 to 16064M 2024-02-07T13:08:49.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:48 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:48 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:49.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:48 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:49.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:48 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:48 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:48 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:48 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.306 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:08:48 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-0[37869]: 2024-02-07T13:08:48.966+0000 7ff2de503700 -1 osd.0 0 waiting for initial osdmap 2024-02-07T13:08:49.306 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:08:48 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-0[37869]: 2024-02-07T13:08:48.970+0000 7ff2da2fd700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-07T13:08:49.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:48 smithi067 ceph-mon[25698]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:49.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:48 smithi067 ceph-mon[25698]: from='osd.0 [v2:172.21.15.67:6802/4131814377,v1:172.21.15.67:6803/4131814377]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-07T13:08:49.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:48 smithi067 ceph-mon[25698]: osdmap e6: 1 total, 0 up, 1 in 2024-02-07T13:08:49.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:48 smithi067 ceph-mon[25698]: from='osd.0 [v2:172.21.15.67:6802/4131814377,v1:172.21.15.67:6803/4131814377]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-07T13:08:49.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:08:49.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:49.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:08:49.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:48 smithi067 ceph-mon[25698]: Adjusting osd_memory_target on smithi067 to 16064M 2024-02-07T13:08:49.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:49.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:49.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:48 smithi159 ceph-mon[27461]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:49.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:48 smithi159 ceph-mon[27461]: from='osd.0 [v2:172.21.15.67:6802/4131814377,v1:172.21.15.67:6803/4131814377]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-07T13:08:49.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:48 smithi159 ceph-mon[27461]: osdmap e6: 1 total, 0 up, 1 in 2024-02-07T13:08:49.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:48 smithi159 ceph-mon[27461]: from='osd.0 [v2:172.21.15.67:6802/4131814377,v1:172.21.15.67:6803/4131814377]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-02-07T13:08:49.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:48 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:08:49.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:48 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:48 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:48 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:08:49.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:48 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:08:49.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:48 smithi159 ceph-mon[27461]: Adjusting osd_memory_target on smithi067 to 16064M 2024-02-07T13:08:49.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:48 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.312 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:48 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:49.312 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:48 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:08:49.312 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:48 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.312 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:48 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.312 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:48 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:49.312 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:48 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:08:50.039 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:49 smithi159 ceph-mon[27461]: from='osd.0 [v2:172.21.15.67:6802/4131814377,v1:172.21.15.67:6803/4131814377]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-02-07T13:08:50.039 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:49 smithi159 ceph-mon[27461]: osdmap e7: 1 total, 0 up, 1 in 2024-02-07T13:08:50.039 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:49 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:08:50.039 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:49 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:08:50.226 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:49 smithi191 ceph-mon[27623]: from='osd.0 [v2:172.21.15.67:6802/4131814377,v1:172.21.15.67:6803/4131814377]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-02-07T13:08:50.226 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:49 smithi191 ceph-mon[27623]: osdmap e7: 1 total, 0 up, 1 in 2024-02-07T13:08:50.226 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:49 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:08:50.226 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:49 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:08:50.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:49 smithi067 ceph-mon[25698]: from='osd.0 [v2:172.21.15.67:6802/4131814377,v1:172.21.15.67:6803/4131814377]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-02-07T13:08:50.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:49 smithi067 ceph-mon[25698]: osdmap e7: 1 total, 0 up, 1 in 2024-02-07T13:08:50.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:49 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:08:50.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:49 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:08:50.841 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping: /dev/nvme4n1 2024-02-07T13:08:50.841 INFO:teuthology.orchestra.run.smithi159.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-07T13:08:50.841 INFO:teuthology.orchestra.run.smithi159.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-07T13:08:50.841 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10+0 records in 2024-02-07T13:08:50.842 INFO:teuthology.orchestra.run.smithi159.stderr:10+0 records out 2024-02-07T13:08:50.842 INFO:teuthology.orchestra.run.smithi159.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129518 s, 810 MB/s 2024-02-07T13:08:50.842 INFO:teuthology.orchestra.run.smithi159.stderr:--> Zapping successful for: 2024-02-07T13:08:51.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:50 smithi159 ceph-mon[27461]: purged_snaps scrub starts 2024-02-07T13:08:51.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:50 smithi159 ceph-mon[27461]: purged_snaps scrub ok 2024-02-07T13:08:51.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:50 smithi159 ceph-mon[27461]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:51.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:08:51.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:50 smithi159 ceph-mon[27461]: osd.0 [v2:172.21.15.67:6802/4131814377,v1:172.21.15.67:6803/4131814377] boot 2024-02-07T13:08:51.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:50 smithi159 ceph-mon[27461]: osdmap e8: 1 total, 1 up, 1 in 2024-02-07T13:08:51.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:08:51.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:50 smithi067 ceph-mon[25698]: purged_snaps scrub starts 2024-02-07T13:08:51.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:50 smithi067 ceph-mon[25698]: purged_snaps scrub ok 2024-02-07T13:08:51.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:50 smithi067 ceph-mon[25698]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:51.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:08:51.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:50 smithi067 ceph-mon[25698]: osd.0 [v2:172.21.15.67:6802/4131814377,v1:172.21.15.67:6803/4131814377] boot 2024-02-07T13:08:51.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:50 smithi067 ceph-mon[25698]: osdmap e8: 1 total, 1 up, 1 in 2024-02-07T13:08:51.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:08:51.366 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:50 smithi191 ceph-mon[27623]: purged_snaps scrub starts 2024-02-07T13:08:51.366 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:50 smithi191 ceph-mon[27623]: purged_snaps scrub ok 2024-02-07T13:08:51.366 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:50 smithi191 ceph-mon[27623]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-07T13:08:51.366 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:08:51.366 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:50 smithi191 ceph-mon[27623]: osd.0 [v2:172.21.15.67:6802/4131814377,v1:172.21.15.67:6803/4131814377] boot 2024-02-07T13:08:51.366 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:50 smithi191 ceph-mon[27623]: osdmap e8: 1 total, 1 up, 1 in 2024-02-07T13:08:51.366 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:08:51.488 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi159:/dev/nvme4n1 2024-02-07T13:08:51.721 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.b/config 2024-02-07T13:08:52.114 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:51 smithi159 ceph-mon[27461]: osdmap e9: 1 total, 1 up, 1 in 2024-02-07T13:08:52.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:51 smithi067 ceph-mon[25698]: osdmap e9: 1 total, 1 up, 1 in 2024-02-07T13:08:52.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:51 smithi191 ceph-mon[27623]: osdmap e9: 1 total, 1 up, 1 in 2024-02-07T13:08:53.012 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:52 smithi159 ceph-mon[27461]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:08:53.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:52 smithi067 ceph-mon[25698]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:08:53.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:52 smithi191 ceph-mon[27623]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:08:54.240 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:53 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T13:08:54.240 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:53 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T13:08:54.240 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:53 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:54.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:53 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T13:08:54.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:53 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T13:08:54.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:53 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:54.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:53 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T13:08:54.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:53 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T13:08:54.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:53 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:08:55.086 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:54 smithi159 ceph-mon[27461]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:08:55.086 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:54 smithi159 ceph-mon[27461]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:08:55.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:54 smithi067 ceph-mon[25698]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:08:55.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:54 smithi067 ceph-mon[25698]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:08:55.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:54 smithi191 ceph-mon[27623]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:08:55.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:54 smithi191 ceph-mon[27623]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi159:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:08:57.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:56 smithi067 ceph-mon[25698]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:08:57.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:56 smithi067 ceph-mon[25698]: from='client.? 172.21.15.159:0/3802638016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24892153-adea-4764-b22f-de70793f3fb4"}]: dispatch 2024-02-07T13:08:57.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:56 smithi067 ceph-mon[25698]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24892153-adea-4764-b22f-de70793f3fb4"}]: dispatch 2024-02-07T13:08:57.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:56 smithi067 ceph-mon[25698]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24892153-adea-4764-b22f-de70793f3fb4"}]': finished 2024-02-07T13:08:57.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:56 smithi067 ceph-mon[25698]: osdmap e10: 2 total, 1 up, 2 in 2024-02-07T13:08:57.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:56 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:08:57.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:56 smithi191 ceph-mon[27623]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:08:57.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:56 smithi191 ceph-mon[27623]: from='client.? 172.21.15.159:0/3802638016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24892153-adea-4764-b22f-de70793f3fb4"}]: dispatch 2024-02-07T13:08:57.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:56 smithi191 ceph-mon[27623]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24892153-adea-4764-b22f-de70793f3fb4"}]: dispatch 2024-02-07T13:08:57.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:56 smithi191 ceph-mon[27623]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24892153-adea-4764-b22f-de70793f3fb4"}]': finished 2024-02-07T13:08:57.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:56 smithi191 ceph-mon[27623]: osdmap e10: 2 total, 1 up, 2 in 2024-02-07T13:08:57.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:08:57.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:56 smithi159 ceph-mon[27461]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:08:57.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:56 smithi159 ceph-mon[27461]: from='client.? 172.21.15.159:0/3802638016' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24892153-adea-4764-b22f-de70793f3fb4"}]: dispatch 2024-02-07T13:08:57.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:56 smithi159 ceph-mon[27461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "24892153-adea-4764-b22f-de70793f3fb4"}]: dispatch 2024-02-07T13:08:57.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:56 smithi159 ceph-mon[27461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "24892153-adea-4764-b22f-de70793f3fb4"}]': finished 2024-02-07T13:08:57.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:56 smithi159 ceph-mon[27461]: osdmap e10: 2 total, 1 up, 2 in 2024-02-07T13:08:57.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:56 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:08:58.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:57 smithi067 ceph-mon[25698]: from='client.? 172.21.15.159:0/3259315945' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T13:08:58.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:57 smithi191 ceph-mon[27623]: from='client.? 172.21.15.159:0/3259315945' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T13:08:58.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:57 smithi159 ceph-mon[27461]: from='client.? 172.21.15.159:0/3259315945' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T13:08:59.277 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:08:58 smithi067 ceph-mon[25698]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:08:59.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:08:58 smithi191 ceph-mon[27623]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:08:59.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:08:58 smithi159 ceph-mon[27461]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:01.122 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:00 smithi159 ceph-mon[27461]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:01.122 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:00 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:01.122 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:00 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:01.123 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:00 smithi159 ceph-mon[27461]: Detected new or changed devices on smithi067 2024-02-07T13:09:01.123 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:00 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:01.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:00 smithi067 ceph-mon[25698]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:01.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:00 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:01.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:00 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:01.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:00 smithi067 ceph-mon[25698]: Detected new or changed devices on smithi067 2024-02-07T13:09:01.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:00 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:01.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:00 smithi191 ceph-mon[27623]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:01.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:00 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:01.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:00 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:01.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:00 smithi191 ceph-mon[27623]: Detected new or changed devices on smithi067 2024-02-07T13:09:01.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:00 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:03.084 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:02 smithi159 ceph-mon[27461]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:03.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:02 smithi067 ceph-mon[25698]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:03.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:02 smithi191 ceph-mon[27623]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:04.012 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:03 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-07T13:09:04.012 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:03 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:04.012 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:03 smithi159 ceph-mon[27461]: Deploying daemon osd.1 on smithi159 2024-02-07T13:09:04.213 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:03 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-07T13:09:04.213 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:03 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:04.213 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:03 smithi067 ceph-mon[25698]: Deploying daemon osd.1 on smithi159 2024-02-07T13:09:04.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:03 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-07T13:09:04.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:03 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:04.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:03 smithi191 ceph-mon[27623]: Deploying daemon osd.1 on smithi159 2024-02-07T13:09:05.040 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:04 smithi067 ceph-mon[25698]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:05.126 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:04 smithi159 ceph-mon[27461]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:05.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:04 smithi191 ceph-mon[27623]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:07.096 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:06 smithi159 ceph-mon[27461]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:07.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:06 smithi067 ceph-mon[25698]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:07.368 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:06 smithi191 ceph-mon[27623]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:08.111 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:08 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:08.111 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:08 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:08.111 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:08 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:08.433 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:08 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:08.433 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:08 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:08.433 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:08 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:08 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:08 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:08 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.098 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:09 smithi159 ceph-mon[27461]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:09.098 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:09 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.098 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:09 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.098 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:09 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.098 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:09 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:09.098 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:09 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:09.098 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:09 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:09.099 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:09 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.099 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:09 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-07T13:09:09.099 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:09 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:09.099 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:09 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.099 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:09 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.099 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:09 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.326 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:09 smithi191 ceph-mon[27623]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:09.326 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:09 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.326 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:09 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.326 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:09 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:09 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:09.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:09 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:09.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:09 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:09.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:09 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:09 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-07T13:09:09.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:09 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:09.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:09 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:09 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.327 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:09 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:09 smithi067 ceph-mon[25698]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:09.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:09.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:09.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:09.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-07T13:09:09.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:09.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:09.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:10.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:10 smithi191 ceph-mon[27623]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:10.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:10 smithi191 ceph-mon[27623]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:10.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:10 smithi191 ceph-mon[27623]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:10.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:10 smithi191 ceph-mon[27623]: Reconfiguring osd.1 (unknown last config time)... 2024-02-07T13:09:10.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:10 smithi191 ceph-mon[27623]: Reconfiguring daemon osd.1 on smithi159 2024-02-07T13:09:10.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:10 smithi191 ceph-mon[27623]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:10.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:10 smithi191 ceph-mon[27623]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:10.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:10 smithi191 ceph-mon[27623]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-02-07T13:09:10.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:10 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:10.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:10 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:10.377 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:10 smithi159 ceph-mon[27461]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:10.378 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:10 smithi159 ceph-mon[27461]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:10.378 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:10 smithi159 ceph-mon[27461]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:10.378 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:10 smithi159 ceph-mon[27461]: Reconfiguring osd.1 (unknown last config time)... 2024-02-07T13:09:10.378 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:10 smithi159 ceph-mon[27461]: Reconfiguring daemon osd.1 on smithi159 2024-02-07T13:09:10.378 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:10 smithi159 ceph-mon[27461]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:10.378 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:10 smithi159 ceph-mon[27461]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:10.378 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:10 smithi159 ceph-mon[27461]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-02-07T13:09:10.378 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:10 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:10.378 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:10 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:10.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:10 smithi067 ceph-mon[25698]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:10.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:10 smithi067 ceph-mon[25698]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:10.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:10 smithi067 ceph-mon[25698]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:10.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:10 smithi067 ceph-mon[25698]: Reconfiguring osd.1 (unknown last config time)... 2024-02-07T13:09:10.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:10 smithi067 ceph-mon[25698]: Reconfiguring daemon osd.1 on smithi159 2024-02-07T13:09:10.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:10 smithi067 ceph-mon[25698]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:10.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:10 smithi067 ceph-mon[25698]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:10.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:10 smithi067 ceph-mon[25698]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-02-07T13:09:10.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:10 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:10.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:10 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:12.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:12 smithi191 ceph-mon[27623]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:12.381 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:12 smithi159 ceph-mon[27461]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:12.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:12 smithi067 ceph-mon[25698]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:13.276 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:13 smithi159 ceph-mon[27461]: from='osd.1 [v2:172.21.15.159:6800/80909837,v1:172.21.15.159:6801/80909837]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-07T13:09:13.277 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:13 smithi159 ceph-mon[27461]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-07T13:09:13.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:13 smithi191 ceph-mon[27623]: from='osd.1 [v2:172.21.15.159:6800/80909837,v1:172.21.15.159:6801/80909837]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-07T13:09:13.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:13 smithi191 ceph-mon[27623]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-07T13:09:13.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:13 smithi067 ceph-mon[25698]: from='osd.1 [v2:172.21.15.159:6800/80909837,v1:172.21.15.159:6801/80909837]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-07T13:09:13.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:13 smithi067 ceph-mon[25698]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-07T13:09:14.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:14 smithi159 ceph-mon[27461]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:14.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:14 smithi159 ceph-mon[27461]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-07T13:09:14.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:14 smithi159 ceph-mon[27461]: from='osd.1 [v2:172.21.15.159:6800/80909837,v1:172.21.15.159:6801/80909837]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-02-07T13:09:14.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:14 smithi159 ceph-mon[27461]: osdmap e11: 2 total, 1 up, 2 in 2024-02-07T13:09:14.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:09:14.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:14 smithi159 ceph-mon[27461]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-02-07T13:09:14.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:14.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:14.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:14.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:14.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:14.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:14.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:14 smithi067 ceph-mon[25698]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:14.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:14 smithi067 ceph-mon[25698]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-07T13:09:14.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:14 smithi067 ceph-mon[25698]: from='osd.1 [v2:172.21.15.159:6800/80909837,v1:172.21.15.159:6801/80909837]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-02-07T13:09:14.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:14 smithi067 ceph-mon[25698]: osdmap e11: 2 total, 1 up, 2 in 2024-02-07T13:09:14.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:09:14.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:14 smithi067 ceph-mon[25698]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-02-07T13:09:14.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:14.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:14.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:14.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:14.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:14.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:14.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:14 smithi191 ceph-mon[27623]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:14.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:14 smithi191 ceph-mon[27623]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-07T13:09:14.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:14 smithi191 ceph-mon[27623]: from='osd.1 [v2:172.21.15.159:6800/80909837,v1:172.21.15.159:6801/80909837]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-02-07T13:09:14.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:14 smithi191 ceph-mon[27623]: osdmap e11: 2 total, 1 up, 2 in 2024-02-07T13:09:14.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:09:14.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:14 smithi191 ceph-mon[27623]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]: dispatch 2024-02-07T13:09:14.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:14.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:14.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:14.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:14.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:14.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:15.005 INFO:teuthology.orchestra.run.smithi159.stdout:Created osd(s) 1 on host 'smithi159' 2024-02-07T13:09:15.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:15 smithi159 ceph-mon[27461]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:15.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:15 smithi159 ceph-mon[27461]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-02-07T13:09:15.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:15 smithi159 ceph-mon[27461]: osdmap e12: 2 total, 1 up, 2 in 2024-02-07T13:09:15.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:15 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:09:15.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:15 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:09:15.311 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:15 smithi159 ceph-mon[27461]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-02-07T13:09:15.312 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:15 smithi159 ceph-mon[27461]: Cluster is now healthy 2024-02-07T13:09:15.312 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:15 smithi159 ceph-mon[27461]: from='osd.1 ' entity='osd.1' 2024-02-07T13:09:15.312 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:15 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:15.312 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:15 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:15.312 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:15 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:15.312 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:15 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:09:15.312 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:15 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:15.312 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:15 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:15.312 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:15 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:15.312 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:15 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:15.312 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:15 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:09:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:15 smithi067 ceph-mon[25698]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:15 smithi067 ceph-mon[25698]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-02-07T13:09:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:15 smithi067 ceph-mon[25698]: osdmap e12: 2 total, 1 up, 2 in 2024-02-07T13:09:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:15 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:09:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:15 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:09:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:15 smithi067 ceph-mon[25698]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-02-07T13:09:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:15 smithi067 ceph-mon[25698]: Cluster is now healthy 2024-02-07T13:09:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:15 smithi067 ceph-mon[25698]: from='osd.1 ' entity='osd.1' 2024-02-07T13:09:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:15 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:15 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:15.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:15 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:15.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:15 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:09:15.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:15 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:15.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:15 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:15.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:15 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:15.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:15 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:15.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:15 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:09:15.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:15 smithi191 ceph-mon[27623]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:15.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:15 smithi191 ceph-mon[27623]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi159", "root=default"]}]': finished 2024-02-07T13:09:15.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:15 smithi191 ceph-mon[27623]: osdmap e12: 2 total, 1 up, 2 in 2024-02-07T13:09:15.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:15 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:09:15.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:15 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:09:15.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:15 smithi191 ceph-mon[27623]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-02-07T13:09:15.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:15 smithi191 ceph-mon[27623]: Cluster is now healthy 2024-02-07T13:09:15.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:15 smithi191 ceph-mon[27623]: from='osd.1 ' entity='osd.1' 2024-02-07T13:09:15.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:15 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:15.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:15 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:15.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:15 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:15.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:15 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:09:15.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:15 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:15.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:15 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:15.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:15 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:15.620 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:15 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:15.621 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:15 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:09:15.622 DEBUG:teuthology.orchestra.run.smithi159:osd.1> sudo journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@osd.1.service 2024-02-07T13:09:15.624 INFO:tasks.cephadm:Deploying osd.2 on smithi191 with /dev/nvme4n1... 2024-02-07T13:09:15.624 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-07T13:09:15.871 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:09:16.240 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:16 smithi191 ceph-mon[27623]: purged_snaps scrub starts 2024-02-07T13:09:16.240 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:16 smithi191 ceph-mon[27623]: purged_snaps scrub ok 2024-02-07T13:09:16.240 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:16 smithi191 ceph-mon[27623]: Adjusting osd_memory_target on smithi159 to 16064M 2024-02-07T13:09:16.240 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:16 smithi191 ceph-mon[27623]: osd.1 [v2:172.21.15.159:6800/80909837,v1:172.21.15.159:6801/80909837] boot 2024-02-07T13:09:16.240 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:16 smithi191 ceph-mon[27623]: osdmap e13: 2 total, 2 up, 2 in 2024-02-07T13:09:16.241 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:16 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:09:16.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:16 smithi159 ceph-mon[27461]: purged_snaps scrub starts 2024-02-07T13:09:16.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:16 smithi159 ceph-mon[27461]: purged_snaps scrub ok 2024-02-07T13:09:16.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:16 smithi159 ceph-mon[27461]: Adjusting osd_memory_target on smithi159 to 16064M 2024-02-07T13:09:16.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:16 smithi159 ceph-mon[27461]: osd.1 [v2:172.21.15.159:6800/80909837,v1:172.21.15.159:6801/80909837] boot 2024-02-07T13:09:16.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:16 smithi159 ceph-mon[27461]: osdmap e13: 2 total, 2 up, 2 in 2024-02-07T13:09:16.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:16 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:09:16.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:16 smithi067 ceph-mon[25698]: purged_snaps scrub starts 2024-02-07T13:09:16.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:16 smithi067 ceph-mon[25698]: purged_snaps scrub ok 2024-02-07T13:09:16.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:16 smithi067 ceph-mon[25698]: Adjusting osd_memory_target on smithi159 to 16064M 2024-02-07T13:09:16.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:16 smithi067 ceph-mon[25698]: osd.1 [v2:172.21.15.159:6800/80909837,v1:172.21.15.159:6801/80909837] boot 2024-02-07T13:09:16.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:16 smithi067 ceph-mon[25698]: osdmap e13: 2 total, 2 up, 2 in 2024-02-07T13:09:16.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:16 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:09:17.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:17 smithi159 ceph-mon[27461]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:17.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:17 smithi067 ceph-mon[25698]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:17.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:17 smithi191 ceph-mon[27623]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-07T13:09:17.692 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping: /dev/nvme4n1 2024-02-07T13:09:17.692 INFO:teuthology.orchestra.run.smithi191.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-07T13:09:17.692 INFO:teuthology.orchestra.run.smithi191.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-07T13:09:17.693 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 10+0 records in 2024-02-07T13:09:17.693 INFO:teuthology.orchestra.run.smithi191.stderr:10+0 records out 2024-02-07T13:09:17.693 INFO:teuthology.orchestra.run.smithi191.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0149133 s, 703 MB/s 2024-02-07T13:09:17.693 INFO:teuthology.orchestra.run.smithi191.stderr: stderr: 2024-02-07T13:09:17.693 INFO:teuthology.orchestra.run.smithi191.stderr:--> Zapping successful for: 2024-02-07T13:09:18.200 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph orch daemon add osd smithi191:/dev/nvme4n1 2024-02-07T13:09:18.226 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:18 smithi191 ceph-mon[27623]: osdmap e14: 2 total, 2 up, 2 in 2024-02-07T13:09:18.434 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:09:18.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:18 smithi159 ceph-mon[27461]: osdmap e14: 2 total, 2 up, 2 in 2024-02-07T13:09:18.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:18 smithi067 ceph-mon[25698]: osdmap e14: 2 total, 2 up, 2 in 2024-02-07T13:09:19.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:19 smithi159 ceph-mon[27461]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:19.479 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:19 smithi191 ceph-mon[27623]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:19.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:19 smithi067 ceph-mon[25698]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:21.168 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:20 smithi191 ceph-mon[27623]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:21.168 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:20 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:21.168 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:20 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:21.168 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:20 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:21.168 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:20 smithi191 ceph-mon[27623]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:09:21.168 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:20 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T13:09:21.168 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:21 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T13:09:21.168 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:21 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:21.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:20 smithi067 ceph-mon[25698]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:21.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:20 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:21.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:20 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:21.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:20 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:21.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:20 smithi067 ceph-mon[25698]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:09:21.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:20 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T13:09:21.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:20 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T13:09:21.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:20 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:21.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:20 smithi159 ceph-mon[27461]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:21.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:20 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:21.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:20 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:21.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:20 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:21.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:20 smithi159 ceph-mon[27461]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi191:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:09:21.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:20 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-07T13:09:21.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:20 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-07T13:09:21.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:21 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:23.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:23 smithi067 ceph-mon[25698]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:23.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:23 smithi067 ceph-mon[25698]: from='client.? 172.21.15.191:0/52401288' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fac69df-3cda-49c6-b775-2822c1e58238"}]: dispatch 2024-02-07T13:09:23.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:23 smithi067 ceph-mon[25698]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fac69df-3cda-49c6-b775-2822c1e58238"}]: dispatch 2024-02-07T13:09:23.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:23 smithi067 ceph-mon[25698]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fac69df-3cda-49c6-b775-2822c1e58238"}]': finished 2024-02-07T13:09:23.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:23 smithi067 ceph-mon[25698]: osdmap e15: 3 total, 2 up, 3 in 2024-02-07T13:09:23.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:23 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:09:23.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:23 smithi191 ceph-mon[27623]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:23.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:23 smithi191 ceph-mon[27623]: from='client.? 172.21.15.191:0/52401288' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fac69df-3cda-49c6-b775-2822c1e58238"}]: dispatch 2024-02-07T13:09:23.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:23 smithi191 ceph-mon[27623]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fac69df-3cda-49c6-b775-2822c1e58238"}]: dispatch 2024-02-07T13:09:23.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:23 smithi191 ceph-mon[27623]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fac69df-3cda-49c6-b775-2822c1e58238"}]': finished 2024-02-07T13:09:23.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:23 smithi191 ceph-mon[27623]: osdmap e15: 3 total, 2 up, 3 in 2024-02-07T13:09:23.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:23 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:09:23.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:23 smithi159 ceph-mon[27461]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:23.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:23 smithi159 ceph-mon[27461]: from='client.? 172.21.15.191:0/52401288' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fac69df-3cda-49c6-b775-2822c1e58238"}]: dispatch 2024-02-07T13:09:23.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:23 smithi159 ceph-mon[27461]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6fac69df-3cda-49c6-b775-2822c1e58238"}]: dispatch 2024-02-07T13:09:23.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:23 smithi159 ceph-mon[27461]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6fac69df-3cda-49c6-b775-2822c1e58238"}]': finished 2024-02-07T13:09:23.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:23 smithi159 ceph-mon[27461]: osdmap e15: 3 total, 2 up, 3 in 2024-02-07T13:09:23.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:23 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:09:25.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:25 smithi067 ceph-mon[25698]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:25.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:25 smithi067 ceph-mon[25698]: from='client.? 172.21.15.191:0/318254060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T13:09:25.368 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:25 smithi191 ceph-mon[27623]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:25.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:25 smithi191 ceph-mon[27623]: from='client.? 172.21.15.191:0/318254060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T13:09:25.399 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:25 smithi159 ceph-mon[27461]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:25.399 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:25 smithi159 ceph-mon[27461]: from='client.? 172.21.15.191:0/318254060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-07T13:09:27.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:27 smithi067 ceph-mon[25698]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:27.368 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:27 smithi191 ceph-mon[27623]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:27.371 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:27 smithi159 ceph-mon[27461]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:29.141 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:29 smithi159 ceph-mon[27461]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:29.141 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:29 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.142 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:29 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.142 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:29 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.142 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:29 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:29.142 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:29 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:29.142 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:29 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:29.142 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:29 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.142 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:29 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.142 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:29 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.142 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:29 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.142 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:29 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:29.142 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:29 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:29.142 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:29 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:29.142 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:29 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.217 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:29 smithi191 ceph-mon[27623]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:29.218 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:29 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.218 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:29 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.218 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:29 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.218 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:29 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:29.218 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:29 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:29.218 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:29 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:29.218 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:29 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.218 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:29 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.218 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:29 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.218 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:29 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.218 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:29 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:29.218 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:29 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:29.219 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:29 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:29.219 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:29 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:29 smithi067 ceph-mon[25698]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:29.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:29 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:29 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:29 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:29 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:29.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:29 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:29.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:29 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:29.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:29 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:29 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:29 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:29 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:29.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:29 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:29.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:29 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:29.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:29 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:29.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:29 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:30.138 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:30 smithi159 ceph-mon[27461]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:30.138 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:30 smithi159 ceph-mon[27461]: Detected new or changed devices on smithi159 2024-02-07T13:09:30.138 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:30 smithi159 ceph-mon[27461]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:30.138 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:30 smithi159 ceph-mon[27461]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:30.139 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:30 smithi159 ceph-mon[27461]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:30.139 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:30 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:30.139 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:30 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:30.139 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:30 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:30.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:30 smithi067 ceph-mon[25698]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:30.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:30 smithi067 ceph-mon[25698]: Detected new or changed devices on smithi159 2024-02-07T13:09:30.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:30 smithi067 ceph-mon[25698]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:30.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:30 smithi067 ceph-mon[25698]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:30.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:30 smithi067 ceph-mon[25698]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:30.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:30 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:30.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:30 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:30.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:30 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:30.367 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:30 smithi191 ceph-mon[27623]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:30.367 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:30 smithi191 ceph-mon[27623]: Detected new or changed devices on smithi159 2024-02-07T13:09:30.367 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:30 smithi191 ceph-mon[27623]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:30.367 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:30 smithi191 ceph-mon[27623]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:30.367 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:30 smithi191 ceph-mon[27623]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:30.367 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:30 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:30.367 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:30 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:30.367 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:30 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:31.125 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:31 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-07T13:09:31.125 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:31 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:31.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:31 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-07T13:09:31.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:31 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:31.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:31 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-07T13:09:31.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:31 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:32.284 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:32 smithi191 ceph-mon[27623]: Deploying daemon osd.2 on smithi191 2024-02-07T13:09:32.285 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:32 smithi191 ceph-mon[27623]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:32.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:32 smithi067 ceph-mon[25698]: Deploying daemon osd.2 on smithi191 2024-02-07T13:09:32.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:32 smithi067 ceph-mon[25698]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:32.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:32 smithi159 ceph-mon[27461]: Deploying daemon osd.2 on smithi191 2024-02-07T13:09:32.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:32 smithi159 ceph-mon[27461]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:34.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:34 smithi067 ceph-mon[25698]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:34.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:34 smithi191 ceph-mon[27623]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:34.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:34 smithi159 ceph-mon[27461]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:36.293 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:36 smithi191 ceph-mon[27623]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:36.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:36 smithi067 ceph-mon[25698]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:36.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:36 smithi159 ceph-mon[27461]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:38.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:38 smithi191 ceph-mon[27623]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:38.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:38 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:38.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:38 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:38.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:38 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:38.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:38 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:38.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:38 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:38.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:38 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:38.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:38 smithi067 ceph-mon[25698]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:38.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:38.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:38.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:38.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:38.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:38.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:38 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:38.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:38 smithi159 ceph-mon[27461]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:38.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:38 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:38.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:38 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:38.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:38 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:38.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:38 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:38.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:38 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:38.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:38 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:40.281 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:40 smithi191 ceph-mon[27623]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:40.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:40 smithi067 ceph-mon[25698]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:40.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:40 smithi159 ceph-mon[27461]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:40.651 INFO:teuthology.orchestra.run.smithi191.stdout:Created osd(s) 2 on host 'smithi191' 2024-02-07T13:09:41.176 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:41 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:41 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:41 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:41 smithi067 ceph-mon[25698]: from='osd.2 [v2:172.21.15.191:6800/2149484224,v1:172.21.15.191:6801/2149484224]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-07T13:09:41.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:41 smithi067 ceph-mon[25698]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-07T13:09:41.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:41 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:41 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:41 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:41.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:41 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:09:41.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:41 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:41 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:41.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:41 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:41.177 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:41 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:41 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:41 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:41 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:41 smithi191 ceph-mon[27623]: from='osd.2 [v2:172.21.15.191:6800/2149484224,v1:172.21.15.191:6801/2149484224]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-07T13:09:41.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:41 smithi191 ceph-mon[27623]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-07T13:09:41.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:41 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:41 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:41 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:41.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:41 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:09:41.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:41 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.260 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:41 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:41.261 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:41 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:41.261 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:41 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.286 DEBUG:teuthology.orchestra.run.smithi191:osd.2> sudo journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@osd.2.service 2024-02-07T13:09:41.329 INFO:tasks.cephadm:Waiting for 3 OSDs to come up... 2024-02-07T13:09:41.329 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph osd stat -f json 2024-02-07T13:09:41.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:41 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:41 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:41 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:41 smithi159 ceph-mon[27461]: from='osd.2 [v2:172.21.15.191:6800/2149484224,v1:172.21.15.191:6801/2149484224]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-07T13:09:41.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:41 smithi159 ceph-mon[27461]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-07T13:09:41.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:41 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:41 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:41 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:41.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:41 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:09:41.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:41 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:41 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:41.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:41 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:41.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:41 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:41.759 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:09:42.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:42 smithi159 ceph-mon[27461]: Adjusting osd_memory_target on smithi191 to 16064M 2024-02-07T13:09:42.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:42 smithi159 ceph-mon[27461]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:42.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:42 smithi159 ceph-mon[27461]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-07T13:09:42.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:42 smithi159 ceph-mon[27461]: from='osd.2 [v2:172.21.15.191:6800/2149484224,v1:172.21.15.191:6801/2149484224]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-07T13:09:42.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:42 smithi159 ceph-mon[27461]: osdmap e16: 3 total, 2 up, 3 in 2024-02-07T13:09:42.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:42 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:09:42.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:42 smithi159 ceph-mon[27461]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-07T13:09:42.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:42 smithi067 ceph-mon[25698]: Adjusting osd_memory_target on smithi191 to 16064M 2024-02-07T13:09:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:42 smithi067 ceph-mon[25698]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:42 smithi067 ceph-mon[25698]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-07T13:09:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:42 smithi067 ceph-mon[25698]: from='osd.2 [v2:172.21.15.191:6800/2149484224,v1:172.21.15.191:6801/2149484224]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-07T13:09:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:42 smithi067 ceph-mon[25698]: osdmap e16: 3 total, 2 up, 3 in 2024-02-07T13:09:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:42 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:09:42.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:42 smithi067 ceph-mon[25698]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-07T13:09:42.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:42 smithi191 ceph-mon[27623]: Adjusting osd_memory_target on smithi191 to 16064M 2024-02-07T13:09:42.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:42 smithi191 ceph-mon[27623]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:42.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:42 smithi191 ceph-mon[27623]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-07T13:09:42.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:42 smithi191 ceph-mon[27623]: from='osd.2 [v2:172.21.15.191:6800/2149484224,v1:172.21.15.191:6801/2149484224]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-07T13:09:42.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:42 smithi191 ceph-mon[27623]: osdmap e16: 3 total, 2 up, 3 in 2024-02-07T13:09:42.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:42 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:09:42.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:42 smithi191 ceph-mon[27623]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]: dispatch 2024-02-07T13:09:43.119 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:09:42 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-2[34299]: 2024-02-07T13:09:42.650+0000 7f9b4ce15700 -1 osd.2 0 waiting for initial osdmap 2024-02-07T13:09:43.119 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:09:42 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-2[34299]: 2024-02-07T13:09:42.655+0000 7f9b44406700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-07T13:09:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:43 smithi067 ceph-mon[25698]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-07T13:09:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:43 smithi067 ceph-mon[25698]: osdmap e17: 3 total, 2 up, 3 in 2024-02-07T13:09:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:43 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:09:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:43 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:09:43.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:43 smithi067 ceph-mon[25698]: from='osd.2 ' entity='osd.2' 2024-02-07T13:09:43.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:43 smithi159 ceph-mon[27461]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-07T13:09:43.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:43 smithi159 ceph-mon[27461]: osdmap e17: 3 total, 2 up, 3 in 2024-02-07T13:09:43.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:43 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:09:43.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:43 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:09:43.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:43 smithi159 ceph-mon[27461]: from='osd.2 ' entity='osd.2' 2024-02-07T13:09:43.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:43 smithi191 ceph-mon[27623]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi191", "root=default"]}]': finished 2024-02-07T13:09:43.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:43 smithi191 ceph-mon[27623]: osdmap e17: 3 total, 2 up, 3 in 2024-02-07T13:09:43.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:43 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:09:43.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:43 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:09:43.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:43 smithi191 ceph-mon[27623]: from='osd.2 ' entity='osd.2' 2024-02-07T13:09:44.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:44 smithi067 ceph-mon[25698]: purged_snaps scrub starts 2024-02-07T13:09:44.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:44 smithi067 ceph-mon[25698]: purged_snaps scrub ok 2024-02-07T13:09:44.249 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:44 smithi067 ceph-mon[25698]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:44.250 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:44 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:09:44.250 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:44 smithi067 ceph-mon[25698]: osd.2 [v2:172.21.15.191:6800/2149484224,v1:172.21.15.191:6801/2149484224] boot 2024-02-07T13:09:44.250 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:44 smithi067 ceph-mon[25698]: osdmap e18: 3 total, 3 up, 3 in 2024-02-07T13:09:44.250 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:44 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:09:44.250 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:09:44.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:44 smithi159 ceph-mon[27461]: purged_snaps scrub starts 2024-02-07T13:09:44.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:44 smithi159 ceph-mon[27461]: purged_snaps scrub ok 2024-02-07T13:09:44.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:44 smithi159 ceph-mon[27461]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:44.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:44 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:09:44.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:44 smithi159 ceph-mon[27461]: osd.2 [v2:172.21.15.191:6800/2149484224,v1:172.21.15.191:6801/2149484224] boot 2024-02-07T13:09:44.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:44 smithi159 ceph-mon[27461]: osdmap e18: 3 total, 3 up, 3 in 2024-02-07T13:09:44.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:44 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:09:44.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:44 smithi191 ceph-mon[27623]: purged_snaps scrub starts 2024-02-07T13:09:44.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:44 smithi191 ceph-mon[27623]: purged_snaps scrub ok 2024-02-07T13:09:44.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:44 smithi191 ceph-mon[27623]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-07T13:09:44.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:44 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:09:44.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:44 smithi191 ceph-mon[27623]: osd.2 [v2:172.21.15.191:6800/2149484224,v1:172.21.15.191:6801/2149484224] boot 2024-02-07T13:09:44.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:44 smithi191 ceph-mon[27623]: osdmap e18: 3 total, 3 up, 3 in 2024-02-07T13:09:44.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:44 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:09:44.840 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":18,"num_osds":3,"num_up_osds":3,"osd_up_since":1707311383,"num_in_osds":3,"osd_in_since":1707311362,"num_remapped_pgs":0} 2024-02-07T13:09:44.840 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-07T13:09:45.080 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:09:45.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:45 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/1232682987' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-07T13:09:45.499 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:45 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/1232682987' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-07T13:09:45.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:45 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/1232682987' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-07T13:09:46.236 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:46 smithi067 ceph-mon[25698]: pgmap v94: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:46.236 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:46 smithi067 ceph-mon[25698]: osdmap e19: 3 total, 3 up, 3 in 2024-02-07T13:09:46.236 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:46 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-07T13:09:46.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:46 smithi159 ceph-mon[27461]: pgmap v94: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:46.462 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:46 smithi159 ceph-mon[27461]: osdmap e19: 3 total, 3 up, 3 in 2024-02-07T13:09:46.462 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:46 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-07T13:09:46.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:46 smithi191 ceph-mon[27623]: pgmap v94: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:46.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:46 smithi191 ceph-mon[27623]: osdmap e19: 3 total, 3 up, 3 in 2024-02-07T13:09:46.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:46 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-07T13:09:46.980 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:09:46.981 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":20,"fsid":"81df3e5a-c5b9-11ee-95b6-87774f69a715","created":"2024-02-07T13:06:11.589197+0000","modified":"2024-02-07T13:09:46.169165+0000","last_up_change":"2024-02-07T13:09:43.655215+0000","last_in_change":"2024-02-07T13:09:22.920752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-07T13:09:45.526400+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e53b131-f387-4600-b27c-e421ff58b822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":4131814377},{"type":"v1","addr":"172.21.15.67:6803","nonce":4131814377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":4131814377},{"type":"v1","addr":"172.21.15.67:6805","nonce":4131814377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":4131814377},{"type":"v1","addr":"172.21.15.67:6809","nonce":4131814377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":4131814377},{"type":"v1","addr":"172.21.15.67:6807","nonce":4131814377}]},"public_addr":"172.21.15.67:6803/4131814377","cluster_addr":"172.21.15.67:6805/4131814377","heartbeat_back_addr":"172.21.15.67:6809/4131814377","heartbeat_front_addr":"172.21.15.67:6807/4131814377","state":["exists","up"]},{"osd":1,"uuid":"24892153-adea-4764-b22f-de70793f3fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":80909837},{"type":"v1","addr":"172.21.15.159:6801","nonce":80909837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":80909837},{"type":"v1","addr":"172.21.15.159:6803","nonce":80909837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":80909837},{"type":"v1","addr":"172.21.15.159:6807","nonce":80909837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":80909837},{"type":"v1","addr":"172.21.15.159:6805","nonce":80909837}]},"public_addr":"172.21.15.159:6801/80909837","cluster_addr":"172.21.15.159:6803/80909837","heartbeat_back_addr":"172.21.15.159:6807/80909837","heartbeat_front_addr":"172.21.15.159:6805/80909837","state":["exists","up"]},{"osd":2,"uuid":"6fac69df-3cda-49c6-b775-2822c1e58238","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2149484224},{"type":"v1","addr":"172.21.15.191:6801","nonce":2149484224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2149484224},{"type":"v1","addr":"172.21.15.191:6803","nonce":2149484224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2149484224},{"type":"v1","addr":"172.21.15.191:6807","nonce":2149484224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2149484224},{"type":"v1","addr":"172.21.15.191:6805","nonce":2149484224}]},"public_addr":"172.21.15.191:6801/2149484224","cluster_addr":"172.21.15.191:6803/2149484224","heartbeat_back_addr":"172.21.15.191:6807/2149484224","heartbeat_front_addr":"172.21.15.191:6805/2149484224","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T13:08:48.327715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T13:09:14.092634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T13:09:41.551859+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/416625014":"2024-02-08T13:07:09.178941+0000","172.21.15.67:6801/1299844367":"2024-02-08T13:07:09.178941+0000","172.21.15.67:6800/1299844367":"2024-02-08T13:07:09.178941+0000","172.21.15.67:0/4116703611":"2024-02-08T13:07:09.178941+0000","172.21.15.67:6800/3665964957":"2024-02-08T13:06:41.781073+0000","172.21.15.67:6801/3665964957":"2024-02-08T13:06:41.781073+0000","172.21.15.67:0/2797980579":"2024-02-08T13:06:41.781073+0000","172.21.15.67:0/3554625152":"2024-02-08T13:06:41.781073+0000","172.21.15.67:0/3615206419":"2024-02-08T13:06:41.781073+0000","172.21.15.67:0/3516969484":"2024-02-08T13:07:09.178941+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-07T13:09:47.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:47 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-07T13:09:47.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:47 smithi067 ceph-mon[25698]: osdmap e20: 3 total, 3 up, 3 in 2024-02-07T13:09:47.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:47 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-07T13:09:47.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:47 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/1892358623' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T13:09:47.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:47 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-07T13:09:47.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:47 smithi159 ceph-mon[27461]: osdmap e20: 3 total, 3 up, 3 in 2024-02-07T13:09:47.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:47 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-07T13:09:47.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:47 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/1892358623' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T13:09:47.618 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:47 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-07T13:09:47.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:47 smithi191 ceph-mon[27623]: osdmap e20: 3 total, 3 up, 3 in 2024-02-07T13:09:47.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:47 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-07T13:09:47.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:47 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/1892358623' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T13:09:47.634 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:09:47 smithi067 sudo[40446]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-07T13:09:47.634 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:09:47 smithi067 sudo[40446]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:09:47.634 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:09:47 smithi067 sudo[40446]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:09:47.634 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:09:47 smithi067 sudo[40446]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:09:47.660 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-07T13:09:45.526400+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 1, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-07T13:09:47.660 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-07T13:09:47.903 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:09:48.025 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:09:47 smithi067 sudo[40467]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-07T13:09:48.025 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:09:47 smithi067 sudo[40467]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:09:48.025 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:09:47 smithi067 sudo[40467]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:09:48.025 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:09:47 smithi067 sudo[40467]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:09:48.277 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:48 smithi067 ceph-mon[25698]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:48.277 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:48 smithi067 ceph-mon[25698]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-07T13:09:48.277 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-07T13:09:48.277 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:48 smithi067 ceph-mon[25698]: osdmap e21: 3 total, 3 up, 3 in 2024-02-07T13:09:48.278 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:48.278 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:48.278 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:48.278 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:48 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:48.319 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:48 smithi159 ceph-mon[27461]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:48.320 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:48 smithi159 ceph-mon[27461]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-07T13:09:48.320 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:48 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-07T13:09:48.320 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:48 smithi159 ceph-mon[27461]: osdmap e21: 3 total, 3 up, 3 in 2024-02-07T13:09:48.320 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:48 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:48.320 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:48 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:48.320 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:48 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:48.320 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:48 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:48.320 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:09:48 smithi159 sudo[34824]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-07T13:09:48.320 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:09:48 smithi159 sudo[34824]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:09:48.320 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:09:48 smithi159 sudo[34824]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:09:48.320 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:09:48 smithi159 sudo[34824]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:09:48.321 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:09:48 smithi159 sudo[34827]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-07T13:09:48.496 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:48 smithi191 ceph-mon[27623]: pgmap v97: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:48.496 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:48 smithi191 ceph-mon[27623]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-07T13:09:48.496 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:48 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-07T13:09:48.496 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:48 smithi191 ceph-mon[27623]: osdmap e21: 3 total, 3 up, 3 in 2024-02-07T13:09:48.496 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:48 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:48.496 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:48 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:48.496 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:48 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:48.496 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:48 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:48.628 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:09:48 smithi159 sudo[34827]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:09:48.628 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:09:48 smithi159 sudo[34827]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:09:48.628 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:09:48 smithi159 sudo[34827]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:09:48.869 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:09:48 smithi191 sudo[35264]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-07T13:09:48.869 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:09:48 smithi191 sudo[35264]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:09:48.869 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:09:48 smithi191 sudo[35264]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:09:48.869 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:09:48 smithi191 sudo[35264]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:09:49.183 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:09:48 smithi191 sudo[35329]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-07T13:09:49.184 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:09:48 smithi191 sudo[35329]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:09:49.184 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:09:48 smithi191 sudo[35329]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:09:49.184 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:09:48 smithi191 sudo[35329]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:09:49.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:49 smithi067 sudo[40593]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-07T13:09:49.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:49 smithi067 sudo[40593]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:09:49.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:49 smithi067 sudo[40593]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:09:49.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:49 smithi067 sudo[40593]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:09:49.455 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:49 smithi191 ceph-mon[27623]: osdmap e22: 3 total, 3 up, 3 in 2024-02-07T13:09:49.455 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:49 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:49.455 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:49 smithi191 ceph-mon[27623]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-07T13:09:49.455 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:49 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:49.455 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:49 smithi191 ceph-mon[27623]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T13:09:49.463 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:49 smithi159 ceph-mon[27461]: osdmap e22: 3 total, 3 up, 3 in 2024-02-07T13:09:49.463 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:49 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:49.463 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:49 smithi159 ceph-mon[27461]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-07T13:09:49.463 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:49 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:49.463 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:49 smithi159 ceph-mon[27461]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T13:09:49.760 INFO:teuthology.orchestra.run.smithi067.stdout:pg_num: 1 2024-02-07T13:09:49.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:49 smithi067 sudo[40616]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-07T13:09:49.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:49 smithi067 sudo[40616]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:09:49.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:49 smithi067 sudo[40616]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:09:49.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:49 smithi067 sudo[40616]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:09:49.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:49 smithi067 ceph-mon[25698]: osdmap e22: 3 total, 3 up, 3 in 2024-02-07T13:09:49.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:49 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:49.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:49 smithi067 ceph-mon[25698]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-07T13:09:49.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:49 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:49.761 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:49 smithi067 ceph-mon[25698]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T13:09:49.984 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:49 smithi191 sudo[35445]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-07T13:09:49.984 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:49 smithi191 sudo[35445]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:09:49.984 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:49 smithi191 sudo[35445]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:09:49.984 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:49 smithi191 sudo[35445]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:09:50.285 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:49 smithi191 sudo[35486]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-07T13:09:50.285 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:49 smithi191 sudo[35486]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:09:50.285 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:49 smithi191 sudo[35486]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:09:50.285 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:49 smithi191 sudo[35486]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:09:50.406 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi067 2024-02-07T13:09:50.407 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi067=foo.a' 2024-02-07T13:09:50.537 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 sudo[34993]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-07T13:09:50.537 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 sudo[34993]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:09:50.537 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 sudo[34993]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:09:50.537 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 sudo[34993]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:09:50.651 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:09:50.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: pgmap v100: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:50.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T13:09:50.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-07T13:09:50.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: Cluster is now healthy 2024-02-07T13:09:50.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:09:50.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:09:50.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:09:50.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:50.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:50.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: Detected new or changed devices on smithi191 2024-02-07T13:09:50.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:50.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:50.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:50.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:50.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:50.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T13:09:50.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:09:50.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:09:50.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:09:50.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: pgmap v101: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:50.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/1236295618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-07T13:09:50.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T13:09:50.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:50.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:50.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:09:50.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:09:50.808 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:50 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:09:50.819 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: pgmap v100: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:50.820 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T13:09:50.820 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-07T13:09:50.820 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: Cluster is now healthy 2024-02-07T13:09:50.820 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:09:50.820 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:09:50.820 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:09:50.820 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:50.820 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:50.820 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: Detected new or changed devices on smithi191 2024-02-07T13:09:50.820 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:50.820 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:50.820 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:50.820 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:50.821 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:50.821 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T13:09:50.821 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:09:50.821 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:09:50.821 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:09:50.821 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: pgmap v101: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:50.821 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/1236295618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-07T13:09:50.821 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T13:09:50.821 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:50.821 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:50.821 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:09:50.821 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:09:50.821 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:50 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:09:50.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 sudo[35033]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-07T13:09:50.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 sudo[35033]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-07T13:09:50.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 sudo[35033]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-07T13:09:50.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 sudo[35033]: pam_unix(sudo:session): session closed for user root 2024-02-07T13:09:50.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: pgmap v100: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:50.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T13:09:50.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-07T13:09:50.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: Cluster is now healthy 2024-02-07T13:09:50.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:09:50.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:09:50.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:09:50.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:50.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:50.829 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: Detected new or changed devices on smithi191 2024-02-07T13:09:50.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:50.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:50.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:50.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:50.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:50.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T13:09:50.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:09:50.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:09:50.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:09:50.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: pgmap v101: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:50.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/1236295618' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-07T13:09:50.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T13:09:50.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:50.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:50.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:09:50.830 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:09:50.831 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:50 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:09:51.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:51 smithi067 ceph-mon[25698]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T13:09:51.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:51 smithi067 ceph-mon[25698]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-07T13:09:51.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:51 smithi067 ceph-mon[25698]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T13:09:51.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:51 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:51.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:51 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:51.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:51 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:51.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:51 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:51.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:51 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:51.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:51 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:51.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:51 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:51.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:51 smithi067 ceph-mon[25698]: pgmap v102: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:51.807 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:51 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:51.833 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:51 smithi191 ceph-mon[27623]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T13:09:51.833 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:51 smithi191 ceph-mon[27623]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-07T13:09:51.833 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:51 smithi191 ceph-mon[27623]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T13:09:51.834 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:51 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:51.834 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:51 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:51.834 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:51 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:51.834 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:51 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:51.834 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:51 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:51.834 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:51 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:51.834 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:51 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:51.834 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:51 smithi191 ceph-mon[27623]: pgmap v102: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:51.834 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:51 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:51.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:51 smithi159 ceph-mon[27461]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-07T13:09:51.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:51 smithi159 ceph-mon[27461]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-07T13:09:51.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:51 smithi159 ceph-mon[27461]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-07T13:09:51.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:51 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:51.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:51 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:51.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:51 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:51.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:51 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:51.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:51 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:51.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:51 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:51.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:51 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:51.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:51 smithi159 ceph-mon[27461]: pgmap v102: 1 pgs: 1 unknown; 0 B data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:51.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:51 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:52.756 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:52 smithi191 ceph-mon[27623]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-02-07T13:09:52.756 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:52 smithi191 ceph-mon[27623]: Cluster is now healthy 2024-02-07T13:09:52.805 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:52 smithi067 ceph-mon[25698]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-02-07T13:09:52.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:52 smithi067 ceph-mon[25698]: Cluster is now healthy 2024-02-07T13:09:52.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:52 smithi159 ceph-mon[27461]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-02-07T13:09:52.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:52 smithi159 ceph-mon[27461]: Cluster is now healthy 2024-02-07T13:09:53.025 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled rgw.foo.a update... 2024-02-07T13:09:53.860 DEBUG:teuthology.orchestra.run.smithi067:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@rgw.foo.a.service 2024-02-07T13:09:53.863 INFO:tasks.cephadm:Adding prometheus.a on smithi159 2024-02-07T13:09:53.864 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph orch apply prometheus '1;smithi159=a' 2024-02-07T13:09:54.102 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:09:54.118 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:54 smithi191 ceph-mon[27623]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:54.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:54 smithi191 ceph-mon[27623]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi067=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:09:54.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:54 smithi191 ceph-mon[27623]: Saving service rgw.foo.a spec with placement smithi067=foo.a;count:1 2024-02-07T13:09:54.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:54 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:54.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:54 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:54.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:54 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:54.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:54 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:54.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:54 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:54.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:54 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-07T13:09:54.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:54 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-07T13:09:54.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:54 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:54.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:54 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:54.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:54 smithi191 ceph-mon[27623]: Deploying daemon rgw.foo.a on smithi067 2024-02-07T13:09:54.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:54 smithi067 ceph-mon[25698]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:54.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:54 smithi067 ceph-mon[25698]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi067=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:09:54.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:54 smithi067 ceph-mon[25698]: Saving service rgw.foo.a spec with placement smithi067=foo.a;count:1 2024-02-07T13:09:54.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:54 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:54.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:54 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:54.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:54 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:54.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:54 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:54.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:54 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:54.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:54 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-07T13:09:54.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:54 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-07T13:09:54.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:54 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:54.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:54 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:54.307 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:54 smithi067 ceph-mon[25698]: Deploying daemon rgw.foo.a on smithi067 2024-02-07T13:09:54.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:54 smithi159 ceph-mon[27461]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:54.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:54 smithi159 ceph-mon[27461]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi067=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:09:54.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:54 smithi159 ceph-mon[27461]: Saving service rgw.foo.a spec with placement smithi067=foo.a;count:1 2024-02-07T13:09:54.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:54 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:54.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:54 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:54.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:54 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:54.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:54 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:54.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:54 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:54.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:54 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-07T13:09:54.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:54 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-07T13:09:54.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:54 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:54.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:54 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:54.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:54 smithi159 ceph-mon[27461]: Deploying daemon rgw.foo.a on smithi067 2024-02-07T13:09:55.306 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:Feb 07 13:09:55 smithi067 systemd[1]: Starting Ceph rgw.foo.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:09:55.575 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:Feb 07 13:09:55 smithi067 podman[41037]: 2024-02-07 13:09:55.364668014 +0000 UTC m=+0.107947665 container create d68d5b811d4b69d833b9e91af9718a5bb8870b4de85d699041a69aeb163ce765 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-rgw-foo-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-07T13:09:55.576 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:Feb 07 13:09:55 smithi067 podman[41037]: 2024-02-07 13:09:55.27469815 +0000 UTC m=+0.017977808 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T13:09:55.576 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:Feb 07 13:09:55 smithi067 podman[41037]: 2024-02-07 13:09:55.575430653 +0000 UTC m=+0.318710306 container init d68d5b811d4b69d833b9e91af9718a5bb8870b4de85d699041a69aeb163ce765 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-rgw-foo-a, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc) 2024-02-07T13:09:55.897 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled prometheus update... 2024-02-07T13:09:56.026 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:Feb 07 13:09:55 smithi067 podman[41037]: 2024-02-07 13:09:55.58145988 +0000 UTC m=+0.324739532 container start d68d5b811d4b69d833b9e91af9718a5bb8870b4de85d699041a69aeb163ce765 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-rgw-foo-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2) 2024-02-07T13:09:56.027 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:Feb 07 13:09:55 smithi067 bash[41037]: d68d5b811d4b69d833b9e91af9718a5bb8870b4de85d699041a69aeb163ce765 2024-02-07T13:09:56.027 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:Feb 07 13:09:55 smithi067 systemd[1]: Started Ceph rgw.foo.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:09:56.202 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:56 smithi191 ceph-mon[27623]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:56.203 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:56.203 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:56.203 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:56.203 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:56.203 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:56.203 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:56 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:56.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:56 smithi067 ceph-mon[25698]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:56.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:56 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:56.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:56 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:56.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:56 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:56.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:56 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:56.293 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:56 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:56.294 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:56 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:56.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:56 smithi159 ceph-mon[27461]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:56.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:56 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:56.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:56 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:56.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:56 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:56.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:56 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:56.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:56 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:56.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:56 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:56.480 DEBUG:teuthology.orchestra.run.smithi159:prometheus.a> sudo journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@prometheus.a.service 2024-02-07T13:09:56.482 INFO:tasks.cephadm:Adding node-exporter.a on smithi067 2024-02-07T13:09:56.482 INFO:tasks.cephadm:Adding node-exporter.b on smithi159 2024-02-07T13:09:56.482 INFO:tasks.cephadm:Adding node-exporter.c on smithi191 2024-02-07T13:09:56.483 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph orch apply node-exporter '3;smithi067=a;smithi159=b;smithi191=c' 2024-02-07T13:09:56.713 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:09:57.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:57 smithi191 ceph-mon[27623]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi159=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:09:57.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:57 smithi191 ceph-mon[27623]: Saving service prometheus spec with placement smithi159=a;count:1 2024-02-07T13:09:57.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:57 smithi191 ceph-mon[27623]: Saving service rgw.foo.a spec with placement smithi067=foo.a;count:1 2024-02-07T13:09:57.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:57 smithi191 ceph-mon[27623]: osdmap e23: 3 total, 3 up, 3 in 2024-02-07T13:09:57.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:57 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/3727735858' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-07T13:09:57.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:57 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:57.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:57 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:57.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:57 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:57.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:57 smithi191 ceph-mon[27623]: Checking dashboard <-> RGW credentials 2024-02-07T13:09:57.157 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:57 smithi067 ceph-mon[25698]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi159=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:09:57.157 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:57 smithi067 ceph-mon[25698]: Saving service prometheus spec with placement smithi159=a;count:1 2024-02-07T13:09:57.157 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:57 smithi067 ceph-mon[25698]: Saving service rgw.foo.a spec with placement smithi067=foo.a;count:1 2024-02-07T13:09:57.157 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:57 smithi067 ceph-mon[25698]: osdmap e23: 3 total, 3 up, 3 in 2024-02-07T13:09:57.157 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:57 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/3727735858' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-07T13:09:57.157 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:57 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:57.157 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:57 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:57.157 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:57 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:57.157 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:57 smithi067 ceph-mon[25698]: Checking dashboard <-> RGW credentials 2024-02-07T13:09:57.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:57 smithi159 ceph-mon[27461]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi159=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:09:57.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:57 smithi159 ceph-mon[27461]: Saving service prometheus spec with placement smithi159=a;count:1 2024-02-07T13:09:57.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:57 smithi159 ceph-mon[27461]: Saving service rgw.foo.a spec with placement smithi067=foo.a;count:1 2024-02-07T13:09:57.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:57 smithi159 ceph-mon[27461]: osdmap e23: 3 total, 3 up, 3 in 2024-02-07T13:09:57.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:57 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/3727735858' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-07T13:09:57.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:57 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:09:57.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:57 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:09:57.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:57 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:09:57.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:57 smithi159 ceph-mon[27461]: Checking dashboard <-> RGW credentials 2024-02-07T13:09:58.073 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:58 smithi191 ceph-mon[27623]: pgmap v106: 33 pgs: 10 creating+peering, 22 unknown, 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:58.073 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:58 smithi191 ceph-mon[27623]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-07T13:09:58.073 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:58 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/3727735858' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-07T13:09:58.073 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:58 smithi191 ceph-mon[27623]: osdmap e24: 3 total, 3 up, 3 in 2024-02-07T13:09:58.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:58 smithi067 ceph-mon[25698]: pgmap v106: 33 pgs: 10 creating+peering, 22 unknown, 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:58.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:58 smithi067 ceph-mon[25698]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-07T13:09:58.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:58 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/3727735858' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-07T13:09:58.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:58 smithi067 ceph-mon[25698]: osdmap e24: 3 total, 3 up, 3 in 2024-02-07T13:09:58.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:58 smithi159 ceph-mon[27461]: pgmap v106: 33 pgs: 10 creating+peering, 22 unknown, 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:09:58.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:58 smithi159 ceph-mon[27461]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-07T13:09:58.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:58 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/3727735858' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-07T13:09:58.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:58 smithi159 ceph-mon[27461]: osdmap e24: 3 total, 3 up, 3 in 2024-02-07T13:09:58.524 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled node-exporter update... 2024-02-07T13:09:59.204 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:59 smithi191 ceph-mon[27623]: osdmap e25: 3 total, 3 up, 3 in 2024-02-07T13:09:59.204 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:59 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-07T13:09:59.204 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:59 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-07T13:09:59.204 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:09:59 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:59.232 DEBUG:teuthology.orchestra.run.smithi067:node-exporter.a> sudo journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@node-exporter.a.service 2024-02-07T13:09:59.235 DEBUG:teuthology.orchestra.run.smithi159:node-exporter.b> sudo journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@node-exporter.b.service 2024-02-07T13:09:59.237 DEBUG:teuthology.orchestra.run.smithi191:node-exporter.c> sudo journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@node-exporter.c.service 2024-02-07T13:09:59.240 INFO:tasks.cephadm:Adding alertmanager.a on smithi067 2024-02-07T13:09:59.240 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph orch apply alertmanager '1;smithi067=a' 2024-02-07T13:09:59.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:59 smithi067 ceph-mon[25698]: osdmap e25: 3 total, 3 up, 3 in 2024-02-07T13:09:59.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:59 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-07T13:09:59.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:59 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-07T13:09:59.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:09:59 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:59.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:59 smithi159 ceph-mon[27461]: osdmap e25: 3 total, 3 up, 3 in 2024-02-07T13:09:59.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:59 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-07T13:09:59.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:59 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-07T13:09:59.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:09:59 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:09:59.514 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:10:00.331 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:00 smithi191 ceph-mon[27623]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi067=a;smithi159=b;smithi191=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:10:00.331 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:00 smithi191 ceph-mon[27623]: Saving service node-exporter spec with placement smithi067=a;smithi159=b;smithi191=c;count:3 2024-02-07T13:10:00.331 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:00 smithi191 ceph-mon[27623]: pgmap v109: 65 pgs: 10 creating+peering, 54 unknown, 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:00.332 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:00 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-07T13:10:00.332 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:00 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-07T13:10:00.332 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:00 smithi191 ceph-mon[27623]: osdmap e26: 3 total, 3 up, 3 in 2024-02-07T13:10:00.332 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:00 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:00.332 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:00 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:00.332 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:00 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:00.332 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:00 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:00.332 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:00 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:00.332 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:00 smithi191 ceph-mon[27623]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-02-07T13:10:00.332 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:00 smithi191 ceph-mon[27623]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-07T13:10:00.332 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:00 smithi191 ceph-mon[27623]: application not enabled on pool 'default.rgw.log' 2024-02-07T13:10:00.332 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:00 smithi191 ceph-mon[27623]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-07T13:10:00.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:00 smithi159 ceph-mon[27461]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi067=a;smithi159=b;smithi191=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:10:00.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:00 smithi159 ceph-mon[27461]: Saving service node-exporter spec with placement smithi067=a;smithi159=b;smithi191=c;count:3 2024-02-07T13:10:00.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:00 smithi159 ceph-mon[27461]: pgmap v109: 65 pgs: 10 creating+peering, 54 unknown, 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:00.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:00 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-07T13:10:00.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:00 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-07T13:10:00.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:00 smithi159 ceph-mon[27461]: osdmap e26: 3 total, 3 up, 3 in 2024-02-07T13:10:00.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:00 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:00.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:00 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:00.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:00 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:00.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:00 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:00.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:00 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:00.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:00 smithi159 ceph-mon[27461]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-02-07T13:10:00.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:00 smithi159 ceph-mon[27461]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-07T13:10:00.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:00 smithi159 ceph-mon[27461]: application not enabled on pool 'default.rgw.log' 2024-02-07T13:10:00.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:00 smithi159 ceph-mon[27461]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-07T13:10:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:00 smithi067 ceph-mon[25698]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi067=a;smithi159=b;smithi191=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:10:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:00 smithi067 ceph-mon[25698]: Saving service node-exporter spec with placement smithi067=a;smithi159=b;smithi191=c;count:3 2024-02-07T13:10:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:00 smithi067 ceph-mon[25698]: pgmap v109: 65 pgs: 10 creating+peering, 54 unknown, 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:00 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-07T13:10:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:00 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-07T13:10:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:00 smithi067 ceph-mon[25698]: osdmap e26: 3 total, 3 up, 3 in 2024-02-07T13:10:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:00 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:00 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:00.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:00 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:00.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:00 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:00.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:00 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:00.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:00 smithi067 ceph-mon[25698]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-02-07T13:10:00.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:00 smithi067 ceph-mon[25698]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-07T13:10:00.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:00 smithi067 ceph-mon[25698]: application not enabled on pool 'default.rgw.log' 2024-02-07T13:10:00.557 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:00 smithi067 ceph-mon[25698]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-07T13:10:01.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:01 smithi191 ceph-mon[27623]: osdmap e27: 3 total, 3 up, 3 in 2024-02-07T13:10:01.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:01 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-07T13:10:01.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:01 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-07T13:10:01.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:01 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-07T13:10:01.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:01 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-07T13:10:01.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:01 smithi191 ceph-mon[27623]: osdmap e28: 3 total, 3 up, 3 in 2024-02-07T13:10:01.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:01 smithi159 ceph-mon[27461]: osdmap e27: 3 total, 3 up, 3 in 2024-02-07T13:10:01.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:01 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-07T13:10:01.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:01 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-07T13:10:01.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:01 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-07T13:10:01.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:01 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-07T13:10:01.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:01 smithi159 ceph-mon[27461]: osdmap e28: 3 total, 3 up, 3 in 2024-02-07T13:10:01.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:01 smithi067 ceph-mon[25698]: osdmap e27: 3 total, 3 up, 3 in 2024-02-07T13:10:01.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:01 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-07T13:10:01.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:01 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-07T13:10:01.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:01 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-07T13:10:01.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:01 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-07T13:10:01.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:01 smithi067 ceph-mon[25698]: osdmap e28: 3 total, 3 up, 3 in 2024-02-07T13:10:01.620 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled alertmanager update... 2024-02-07T13:10:02.213 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:02 smithi191 ceph-mon[27623]: pgmap v112: 97 pgs: 10 creating+peering, 63 unknown, 24 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-07T13:10:02.213 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:02 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:02.213 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:02 smithi191 ceph-mon[27623]: osdmap e29: 3 total, 3 up, 3 in 2024-02-07T13:10:02.241 DEBUG:teuthology.orchestra.run.smithi067:alertmanager.a> sudo journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@alertmanager.a.service 2024-02-07T13:10:02.244 INFO:tasks.cephadm:Adding grafana.a on smithi159 2024-02-07T13:10:02.244 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph orch apply grafana '1;smithi159=a' 2024-02-07T13:10:02.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:02 smithi159 ceph-mon[27461]: pgmap v112: 97 pgs: 10 creating+peering, 63 unknown, 24 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-07T13:10:02.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:02 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:02.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:02 smithi159 ceph-mon[27461]: osdmap e29: 3 total, 3 up, 3 in 2024-02-07T13:10:02.481 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:10:02.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:02 smithi067 ceph-mon[25698]: pgmap v112: 97 pgs: 10 creating+peering, 63 unknown, 24 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-02-07T13:10:02.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:02 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:02.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:02 smithi067 ceph-mon[25698]: osdmap e29: 3 total, 3 up, 3 in 2024-02-07T13:10:03.354 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:03 smithi191 ceph-mon[27623]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi067=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:10:03.354 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:03 smithi191 ceph-mon[27623]: Saving service alertmanager spec with placement smithi067=a;count:1 2024-02-07T13:10:03.354 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:03 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-07T13:10:03.354 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:03 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-07T13:10:03.354 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:03 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-07T13:10:03.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:03 smithi159 ceph-mon[27461]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi067=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:10:03.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:03 smithi159 ceph-mon[27461]: Saving service alertmanager spec with placement smithi067=a;count:1 2024-02-07T13:10:03.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:03 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-07T13:10:03.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:03 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-07T13:10:03.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:03 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-07T13:10:03.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:03 smithi067 ceph-mon[25698]: from='client.24268 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi067=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:10:03.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:03 smithi067 ceph-mon[25698]: Saving service alertmanager spec with placement smithi067=a;count:1 2024-02-07T13:10:03.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:03 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-07T13:10:03.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:03 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-07T13:10:03.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:03 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-07T13:10:04.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:04 smithi191 ceph-mon[27623]: pgmap v115: 129 pgs: 11 creating+peering, 44 unknown, 74 active+clean; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-07T13:10:04.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:04 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-07T13:10:04.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:04 smithi191 ceph-mon[27623]: osdmap e30: 3 total, 3 up, 3 in 2024-02-07T13:10:04.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:04 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-07T13:10:04.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:04 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-07T13:10:04.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:04 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-07T13:10:04.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:04 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-07T13:10:04.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:04 smithi191 ceph-mon[27623]: osdmap e31: 3 total, 3 up, 3 in 2024-02-07T13:10:04.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:04 smithi159 ceph-mon[27461]: pgmap v115: 129 pgs: 11 creating+peering, 44 unknown, 74 active+clean; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-07T13:10:04.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:04 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-07T13:10:04.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:04 smithi159 ceph-mon[27461]: osdmap e30: 3 total, 3 up, 3 in 2024-02-07T13:10:04.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:04 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-07T13:10:04.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:04 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-07T13:10:04.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:04 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-07T13:10:04.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:04 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-07T13:10:04.444 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:04 smithi159 ceph-mon[27461]: osdmap e31: 3 total, 3 up, 3 in 2024-02-07T13:10:04.448 INFO:teuthology.orchestra.run.smithi191.stdout:Scheduled grafana update... 2024-02-07T13:10:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:04 smithi067 ceph-mon[25698]: pgmap v115: 129 pgs: 11 creating+peering, 44 unknown, 74 active+clean; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.7 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-02-07T13:10:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:04 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-07T13:10:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:04 smithi067 ceph-mon[25698]: osdmap e30: 3 total, 3 up, 3 in 2024-02-07T13:10:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:04 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-07T13:10:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:04 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-07T13:10:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:04 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/1665157172' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-07T13:10:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:04 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/3684936006' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-07T13:10:04.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:04 smithi067 ceph-mon[25698]: osdmap e31: 3 total, 3 up, 3 in 2024-02-07T13:10:05.091 DEBUG:teuthology.orchestra.run.smithi159:grafana.a> sudo journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@grafana.a.service 2024-02-07T13:10:05.094 INFO:tasks.cephadm:Setting up client nodes... 2024-02-07T13:10:05.094 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-07T13:10:05.342 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:10:05.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:05 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:05.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:05 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:05.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:05 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:05.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:05 smithi191 ceph-mon[27623]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi159=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:10:05.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:05 smithi191 ceph-mon[27623]: Saving service grafana spec with placement smithi159=a;count:1 2024-02-07T13:10:05.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:05 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:05.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:05 smithi191 ceph-mon[27623]: Deploying daemon prometheus.a on smithi159 2024-02-07T13:10:05.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:05 smithi191 ceph-mon[27623]: pgmap v118: 129 pgs: 11 creating+peering, 23 unknown, 95 active+clean; 579 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 5.2 KiB/s rd, 4.0 KiB/s wr, 22 op/s 2024-02-07T13:10:05.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:05 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:05.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:05 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:05.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:05 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:05.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:05 smithi067 ceph-mon[25698]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi159=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:10:05.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:05 smithi067 ceph-mon[25698]: Saving service grafana spec with placement smithi159=a;count:1 2024-02-07T13:10:05.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:05 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:05.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:05 smithi067 ceph-mon[25698]: Deploying daemon prometheus.a on smithi159 2024-02-07T13:10:05.464 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:05 smithi067 ceph-mon[25698]: pgmap v118: 129 pgs: 11 creating+peering, 23 unknown, 95 active+clean; 579 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 5.2 KiB/s rd, 4.0 KiB/s wr, 22 op/s 2024-02-07T13:10:05.650 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:05 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:05.650 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:05 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:05.650 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:05 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:05.650 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:05 smithi159 ceph-mon[27461]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi159=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:10:05.650 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:05 smithi159 ceph-mon[27461]: Saving service grafana spec with placement smithi159=a;count:1 2024-02-07T13:10:05.650 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:05 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:05.650 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:05 smithi159 ceph-mon[27461]: Deploying daemon prometheus.a on smithi159 2024-02-07T13:10:05.650 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:05 smithi159 ceph-mon[27461]: pgmap v118: 129 pgs: 11 creating+peering, 23 unknown, 95 active+clean; 579 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 5.2 KiB/s rd, 4.0 KiB/s wr, 22 op/s 2024-02-07T13:10:06.492 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:06 smithi067 ceph-mon[25698]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-07T13:10:06.492 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:06 smithi067 ceph-mon[25698]: Cluster is now healthy 2024-02-07T13:10:06.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:06 smithi191 ceph-mon[27623]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-07T13:10:06.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:06 smithi191 ceph-mon[27623]: Cluster is now healthy 2024-02-07T13:10:06.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:06 smithi159 ceph-mon[27461]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-02-07T13:10:06.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:06 smithi159 ceph-mon[27461]: Cluster is now healthy 2024-02-07T13:10:07.349 INFO:teuthology.orchestra.run.smithi067.stdout:[client.0] 2024-02-07T13:10:07.349 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAvgcNlmRGwFBAA/0jl+2mhpaMdt2m3cAbD4Q== 2024-02-07T13:10:07.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:07 smithi067 ceph-mon[25698]: pgmap v119: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 64 KiB/s rd, 6.7 KiB/s wr, 161 op/s 2024-02-07T13:10:07.618 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:07 smithi191 ceph-mon[27623]: pgmap v119: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 64 KiB/s rd, 6.7 KiB/s wr, 161 op/s 2024-02-07T13:10:07.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:07 smithi159 ceph-mon[27461]: pgmap v119: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 64 KiB/s rd, 6.7 KiB/s wr, 161 op/s 2024-02-07T13:10:07.924 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-07T13:10:07.924 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-07T13:10:07.924 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-07T13:10:07.967 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-07T13:10:08.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:08 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/301173241' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:10:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:08 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/301173241' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T13:10:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:08 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:08 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:08.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:08 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:08.584 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:08 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/301173241' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:10:08.584 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:08 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/301173241' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T13:10:08.585 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:08 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:08.585 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:08 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:08.585 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:08 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:08.618 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:08 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/301173241' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:10:08.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:08 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/301173241' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T13:10:08.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:08 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:08.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:08 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:08.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:08 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:10.118 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:09 smithi191 ceph-mon[27623]: pgmap v120: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 53 KiB/s rd, 5.4 KiB/s wr, 135 op/s 2024-02-07T13:10:10.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:09 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:10.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:09 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:10.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:09 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:10.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:09 smithi159 ceph-mon[27461]: pgmap v120: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 53 KiB/s rd, 5.4 KiB/s wr, 135 op/s 2024-02-07T13:10:10.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:09 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:10.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:09 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:10.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:09 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:10.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:09 smithi067 ceph-mon[25698]: pgmap v120: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 53 KiB/s rd, 5.4 KiB/s wr, 135 op/s 2024-02-07T13:10:10.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:10.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:10.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:09 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:11.547 INFO:teuthology.orchestra.run.smithi159.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.b/config 2024-02-07T13:10:11.930 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:11 smithi159 ceph-mon[27461]: pgmap v121: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 140 op/s 2024-02-07T13:10:12.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:11 smithi191 ceph-mon[27623]: pgmap v121: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 140 op/s 2024-02-07T13:10:12.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:11 smithi067 ceph-mon[25698]: pgmap v121: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 140 op/s 2024-02-07T13:10:14.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:13 smithi191 ceph-mon[27623]: pgmap v122: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 49 KiB/s rd, 3.7 KiB/s wr, 118 op/s 2024-02-07T13:10:14.191 INFO:teuthology.orchestra.run.smithi159.stdout:[client.1] 2024-02-07T13:10:14.191 INFO:teuthology.orchestra.run.smithi159.stdout: key = AQA2gcNlyvQfCxAALmSKOBXG+wdnPsmNqfNNMQ== 2024-02-07T13:10:14.192 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:13 smithi159 ceph-mon[27461]: pgmap v122: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 49 KiB/s rd, 3.7 KiB/s wr, 118 op/s 2024-02-07T13:10:14.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:13 smithi067 ceph-mon[25698]: pgmap v122: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 49 KiB/s rd, 3.7 KiB/s wr, 118 op/s 2024-02-07T13:10:15.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:14 smithi191 ceph-mon[27623]: from='client.? 172.21.15.159:0/1309949693' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:10:15.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:14 smithi191 ceph-mon[27623]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:10:15.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:14 smithi191 ceph-mon[27623]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T13:10:15.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:14 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:15.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:14 smithi159 ceph-mon[27461]: from='client.? 172.21.15.159:0/1309949693' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:10:15.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:14 smithi159 ceph-mon[27461]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:10:15.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:14 smithi159 ceph-mon[27461]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T13:10:15.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:14 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:15.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:14 smithi067 ceph-mon[25698]: from='client.? 172.21.15.159:0/1309949693' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:10:15.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:14 smithi067 ceph-mon[25698]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:10:15.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:14 smithi067 ceph-mon[25698]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T13:10:15.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:14 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:16.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:16 smithi067 ceph-mon[25698]: pgmap v123: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 44 KiB/s rd, 3.3 KiB/s wr, 106 op/s 2024-02-07T13:10:16.618 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:16 smithi191 ceph-mon[27623]: pgmap v123: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 44 KiB/s rd, 3.3 KiB/s wr, 106 op/s 2024-02-07T13:10:16.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:16 smithi159 ceph-mon[27461]: pgmap v123: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 44 KiB/s rd, 3.3 KiB/s wr, 106 op/s 2024-02-07T13:10:17.226 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-02-07T13:10:17.227 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-07T13:10:17.227 DEBUG:teuthology.orchestra.run.smithi159:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-07T13:10:17.273 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph auth get-or-create client.2 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-07T13:10:17.523 INFO:teuthology.orchestra.run.smithi191.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.c/config 2024-02-07T13:10:18.329 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:18 smithi191 ceph-mon[27623]: pgmap v124: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 37 KiB/s rd, 1.9 KiB/s wr, 87 op/s 2024-02-07T13:10:18.459 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-mon[27461]: pgmap v124: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 37 KiB/s rd, 1.9 KiB/s wr, 87 op/s 2024-02-07T13:10:18.460 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 systemd[1]: Starting Ceph prometheus.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:10:18.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:18 smithi067 ceph-mon[25698]: pgmap v124: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 37 KiB/s rd, 1.9 KiB/s wr, 87 op/s 2024-02-07T13:10:18.710 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 podman[36025]: 2024-02-07 13:10:18.362471195 +0000 UTC m=+0.017858917 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-07T13:10:18.710 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 podman[36025]: 2024-02-07 13:10:18.475561667 +0000 UTC m=+0.130949432 container create 0ee24705a6619775c313bc831ea573497aa3daacd376710856f4fdba46299acf (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-07T13:10:18.710 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 podman[36025]: 2024-02-07 13:10:18.677810754 +0000 UTC m=+0.333198479 container init 0ee24705a6619775c313bc831ea573497aa3daacd376710856f4fdba46299acf (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-07T13:10:18.710 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 podman[36025]: 2024-02-07 13:10:18.68357146 +0000 UTC m=+0.338959184 container start 0ee24705a6619775c313bc831ea573497aa3daacd376710856f4fdba46299acf (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-07T13:10:18.711 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.706Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-07T13:10:18.711 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.706Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-07T13:10:18.711 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.706Z caller=main.go:570 level=info host_details="(Linux 5.14.0-412.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Jan 24 21:50:18 UTC 2024 x86_64 smithi159 (none))" 2024-02-07T13:10:18.711 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.707Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-07T13:10:18.711 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.707Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-07T13:10:18.711 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.708Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-07T13:10:18.711 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.709Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-07T13:10:18.995 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.709Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-07T13:10:18.995 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.710Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-07T13:10:18.995 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.712Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-07T13:10:18.996 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.712Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.607µs 2024-02-07T13:10:18.996 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.712Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-07T13:10:18.996 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.712Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-07T13:10:18.996 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.712Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=18.54µs wal_replay_duration=209.542µs wbl_replay_duration=85ns total_replay_duration=245.701µs 2024-02-07T13:10:18.996 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.713Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-07T13:10:18.996 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.713Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-07T13:10:18.996 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.713Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-07T13:10:18.996 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.729Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=16.300868ms db_storage=821ns remote_storage=1.087µs web_handler=320ns query_engine=589ns scrape=374.839µs scrape_sd=69.46µs notify=927ns notify_sd=2.138µs rules=15.678ms tracing=7.677µs 2024-02-07T13:10:18.996 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.729Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-07T13:10:18.996 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:10:18.729Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-07T13:10:18.996 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 bash[36025]: 0ee24705a6619775c313bc831ea573497aa3daacd376710856f4fdba46299acf 2024-02-07T13:10:18.996 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:10:18 smithi159 systemd[1]: Started Ceph prometheus.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:10:19.471 INFO:teuthology.orchestra.run.smithi191.stdout:[client.2] 2024-02-07T13:10:19.471 INFO:teuthology.orchestra.run.smithi191.stdout: key = AQA7gcNlfJHQGxAA+/CpVHjKh/xGVavemG1UJQ== 2024-02-07T13:10:20.094 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:19 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: ignoring --setuser ceph since I am not root 2024-02-07T13:10:20.094 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:19 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: ignoring --setgroup ceph since I am not root 2024-02-07T13:10:20.094 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:19 smithi159 ceph-mgr[29175]: -- 172.21.15.159:0/1182305570 <== mon.2 v2:172.21.15.159:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e1dd22b520 con 0x55e1dd200800 2024-02-07T13:10:20.094 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:19 smithi159 ceph-mon[27461]: pgmap v125: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 9.7 KiB/s rd, 0 B/s wr, 19 op/s 2024-02-07T13:10:20.094 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:19 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.095 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:19 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.095 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:19 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.095 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:19 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-07T13:10:20.095 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:19 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.095 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:19 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.095 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:19 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.095 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:19 smithi159 ceph-mon[27461]: from='client.? 172.21.15.191:0/3964322617' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:10:20.095 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:19 smithi159 ceph-mon[27461]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:10:20.095 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:19 smithi159 ceph-mon[27461]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T13:10:20.095 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:19 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.118 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:19 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: ignoring --setuser ceph since I am not root 2024-02-07T13:10:20.118 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:19 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: ignoring --setgroup ceph since I am not root 2024-02-07T13:10:20.119 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:19 smithi191 ceph-mgr[30125]: -- 172.21.15.191:0/956295344 <== mon.0 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55eb2eeb9520 con 0x55eb2ee8e800 2024-02-07T13:10:20.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:19 smithi191 ceph-mon[27623]: pgmap v125: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 9.7 KiB/s rd, 0 B/s wr, 19 op/s 2024-02-07T13:10:20.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:19 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:19 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:19 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:19 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-07T13:10:20.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:19 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:19 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:19 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:19 smithi191 ceph-mon[27623]: from='client.? 172.21.15.191:0/3964322617' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:10:20.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:19 smithi191 ceph-mon[27623]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:10:20.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:19 smithi191 ceph-mon[27623]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T13:10:20.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:19 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.152 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-07T13:10:20.152 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/etc/ceph/ceph.client.2.keyring 2024-02-07T13:10:20.152 DEBUG:teuthology.orchestra.run.smithi191:> sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-02-07T13:10:20.196 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-07T13:10:20.196 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-07T13:10:20.197 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph mgr dump --format=json 2024-02-07T13:10:20.223 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:19 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: ignoring --setuser ceph since I am not root 2024-02-07T13:10:20.223 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:19 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: ignoring --setgroup ceph since I am not root 2024-02-07T13:10:20.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:19 smithi067 ceph-mon[25698]: pgmap v125: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 9.7 KiB/s rd, 0 B/s wr, 19 op/s 2024-02-07T13:10:20.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-07T13:10:20.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:19 smithi067 ceph-mon[25698]: from='client.? 172.21.15.191:0/3964322617' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:10:20.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:19 smithi067 ceph-mon[25698]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-07T13:10:20.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:19 smithi067 ceph-mon[25698]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-07T13:10:20.224 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:19 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' 2024-02-07T13:10:20.444 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:10:20.532 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:20 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:20.428+0000 7ff36233a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T13:10:20.532 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:20 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:20.528+0000 7ff36233a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T13:10:20.693 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:20 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:20.411+0000 7fd66543d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T13:10:20.693 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:20 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:20.506+0000 7fd66543d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T13:10:20.843 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:20 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:20.421+0000 7f0ad848a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-07T13:10:20.843 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:20 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:20.519+0000 7f0ad848a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-07T13:10:21.118 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:20 smithi191 ceph-mon[27623]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-07T13:10:21.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:20 smithi191 ceph-mon[27623]: mgrmap e17: y(active, since 3m), standbys: z, x 2024-02-07T13:10:21.142 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:20 smithi159 ceph-mon[27461]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-07T13:10:21.143 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:20 smithi159 ceph-mon[27461]: mgrmap e17: y(active, since 3m), standbys: z, x 2024-02-07T13:10:21.203 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:20 smithi067 ceph-mon[25698]: from='mgr.14152 172.21.15.67:0/2161134070' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-07T13:10:21.203 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:20 smithi067 ceph-mon[25698]: mgrmap e17: y(active, since 3m), standbys: z, x 2024-02-07T13:10:21.443 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:21 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:21.142+0000 7fd66543d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T13:10:21.443 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:21 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:21.235+0000 7fd66543d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T13:10:21.443 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:21 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:21.376+0000 7fd66543d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T13:10:21.514 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:21 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:21.180+0000 7f0ad848a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T13:10:21.514 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:21 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:21.275+0000 7f0ad848a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T13:10:21.514 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:21 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:21.419+0000 7f0ad848a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T13:10:21.514 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:21 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:21.513+0000 7f0ad848a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T13:10:21.542 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:21 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:21.202+0000 7ff36233a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-07T13:10:21.542 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:21 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:21.297+0000 7ff36233a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-07T13:10:21.542 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:21 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:21.443+0000 7ff36233a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-07T13:10:21.542 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:21 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:21.540+0000 7ff36233a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T13:10:21.806 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:21 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:21.626+0000 7ff36233a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T13:10:21.818 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:21 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:21.472+0000 7fd66543d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-07T13:10:21.818 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:21 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:21.555+0000 7fd66543d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T13:10:21.858 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:21 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:21.596+0000 7f0ad848a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-07T13:10:21.858 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:21 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:21.857+0000 7f0ad848a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T13:10:22.109 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:21 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:21.817+0000 7fd66543d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T13:10:22.109 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:21 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:21.927+0000 7fd66543d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T13:10:22.110 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:22 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:22.014+0000 7fd66543d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T13:10:22.110 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:22 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:22.108+0000 7fd66543d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T13:10:22.119 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:21 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:21.966+0000 7f0ad848a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T13:10:22.119 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:22 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:22.051+0000 7f0ad848a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T13:10:22.195 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:21 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:21.898+0000 7ff36233a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-07T13:10:22.195 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:22 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:22.013+0000 7ff36233a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-07T13:10:22.196 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:22 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:22.108+0000 7ff36233a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-07T13:10:22.443 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:22 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:22.284+0000 7fd66543d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T13:10:22.555 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:22 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:22.206+0000 7ff36233a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T13:10:22.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:22 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:22.389+0000 7ff36233a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T13:10:22.583 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:10:22.618 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:22 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:22.145+0000 7f0ad848a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-07T13:10:22.618 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:22 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:22.320+0000 7f0ad848a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-07T13:10:22.889 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:22 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/3199839878' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-07T13:10:22.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:22 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/3199839878' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-07T13:10:23.118 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:22 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/3199839878' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-07T13:10:23.210 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":3886435222},{"type":"v1","addr":"172.21.15.67:6801","nonce":3886435222}]},"active_addr":"172.21.15.67:6801/3886435222","active_change":"2024-02-07T13:07:09.179048+0000","active_mgr_features":4540701272860131327,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"x","mgr_features":4540701272860131327,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.67:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":3779372659}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":844469757}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":3548590016}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.67:0","nonce":3739908062}]}]} 2024-02-07T13:10:23.215 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-07T13:10:23.216 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-07T13:10:23.216 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-07T13:10:23.458 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:10:23.547 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:23 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:23.400+0000 7ff36233a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T13:10:23.547 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:23 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:23.546+0000 7ff36233a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T13:10:23.618 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:23 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:23.276+0000 7f0ad848a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T13:10:23.619 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:23 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:23.406+0000 7f0ad848a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T13:10:23.693 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:23 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:23.258+0000 7fd66543d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-07T13:10:23.693 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:23 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:23.389+0000 7fd66543d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-07T13:10:24.012 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:23 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:23.724+0000 7fd66543d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T13:10:24.013 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:23 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:23.815+0000 7fd66543d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T13:10:24.013 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:24 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:24.011+0000 7fd66543d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:10:24.023 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:23 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:23.740+0000 7f0ad848a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T13:10:24.023 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:23 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:23.829+0000 7f0ad848a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T13:10:24.023 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:24 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:24.022+0000 7f0ad848a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:10:24.191 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:23 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:23.894+0000 7ff36233a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-07T13:10:24.191 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:23 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:23.988+0000 7ff36233a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-07T13:10:24.191 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:24 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:24.190+0000 7ff36233a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:10:24.368 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:24 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:24.206+0000 7f0ad848a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T13:10:24.424 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:24 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:24.207+0000 7fd66543d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T13:10:24.555 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:24 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:24.383+0000 7ff36233a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-07T13:10:24.693 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:24 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:24.512+0000 7fd66543d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:10:24.751 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:24 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:24.489+0000 7f0ad848a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:10:24.751 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:24 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:24.750+0000 7f0ad848a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T13:10:24.882 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:24 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:24.685+0000 7ff36233a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-07T13:10:25.118 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:24 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:24.986+0000 7f0ad848a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T13:10:25.189 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:24 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:24.784+0000 7fd66543d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T13:10:25.189 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:25 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:25.026+0000 7fd66543d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T13:10:25.204 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:24 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:24.953+0000 7ff36233a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-07T13:10:25.330 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:10:25.330 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":31,"fsid":"81df3e5a-c5b9-11ee-95b6-87774f69a715","created":"2024-02-07T13:06:11.589197+0000","modified":"2024-02-07T13:10:04.070967+0000","last_up_change":"2024-02-07T13:09:43.655215+0000","last_in_change":"2024-02-07T13:09:22.920752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-07T13:09:45.526400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-07T13:09:55.860713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-02-07T13:09:57.358100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-02-07T13:09:59.320263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-02-07T13:10:01.371824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e53b131-f387-4600-b27c-e421ff58b822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":4131814377},{"type":"v1","addr":"172.21.15.67:6803","nonce":4131814377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":4131814377},{"type":"v1","addr":"172.21.15.67:6805","nonce":4131814377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":4131814377},{"type":"v1","addr":"172.21.15.67:6809","nonce":4131814377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":4131814377},{"type":"v1","addr":"172.21.15.67:6807","nonce":4131814377}]},"public_addr":"172.21.15.67:6803/4131814377","cluster_addr":"172.21.15.67:6805/4131814377","heartbeat_back_addr":"172.21.15.67:6809/4131814377","heartbeat_front_addr":"172.21.15.67:6807/4131814377","state":["exists","up"]},{"osd":1,"uuid":"24892153-adea-4764-b22f-de70793f3fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":80909837},{"type":"v1","addr":"172.21.15.159:6801","nonce":80909837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":80909837},{"type":"v1","addr":"172.21.15.159:6803","nonce":80909837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":80909837},{"type":"v1","addr":"172.21.15.159:6807","nonce":80909837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":80909837},{"type":"v1","addr":"172.21.15.159:6805","nonce":80909837}]},"public_addr":"172.21.15.159:6801/80909837","cluster_addr":"172.21.15.159:6803/80909837","heartbeat_back_addr":"172.21.15.159:6807/80909837","heartbeat_front_addr":"172.21.15.159:6805/80909837","state":["exists","up"]},{"osd":2,"uuid":"6fac69df-3cda-49c6-b775-2822c1e58238","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2149484224},{"type":"v1","addr":"172.21.15.191:6801","nonce":2149484224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2149484224},{"type":"v1","addr":"172.21.15.191:6803","nonce":2149484224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2149484224},{"type":"v1","addr":"172.21.15.191:6807","nonce":2149484224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2149484224},{"type":"v1","addr":"172.21.15.191:6805","nonce":2149484224}]},"public_addr":"172.21.15.191:6801/2149484224","cluster_addr":"172.21.15.191:6803/2149484224","heartbeat_back_addr":"172.21.15.191:6807/2149484224","heartbeat_front_addr":"172.21.15.191:6805/2149484224","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T13:08:48.327715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T13:09:14.092634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T13:09:41.551859+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/416625014":"2024-02-08T13:07:09.178941+0000","172.21.15.67:6801/1299844367":"2024-02-08T13:07:09.178941+0000","172.21.15.67:6800/1299844367":"2024-02-08T13:07:09.178941+0000","172.21.15.67:0/4116703611":"2024-02-08T13:07:09.178941+0000","172.21.15.67:6800/3665964957":"2024-02-08T13:06:41.781073+0000","172.21.15.67:6801/3665964957":"2024-02-08T13:06:41.781073+0000","172.21.15.67:0/2797980579":"2024-02-08T13:06:41.781073+0000","172.21.15.67:0/3554625152":"2024-02-08T13:06:41.781073+0000","172.21.15.67:0/3615206419":"2024-02-08T13:06:41.781073+0000","172.21.15.67:0/3516969484":"2024-02-08T13:07:09.178941+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-07T13:10:25.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:25 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/127476819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T13:10:25.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:25 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/127476819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T13:10:25.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:25 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:25.203+0000 7ff36233a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-07T13:10:25.779 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:25 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/127476819' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T13:10:25.957 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-07T13:10:25.957 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph osd dump --format=json 2024-02-07T13:10:26.118 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:25 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:25.778+0000 7f0ad848a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T13:10:26.193 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:25 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:25.835+0000 7fd66543d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T13:10:26.203 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:10:26.305 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:26 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:26.047+0000 7ff36233a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-07T13:10:26.618 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:26 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:26.348+0000 7f0ad848a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T13:10:26.619 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:26 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:26.446+0000 7f0ad848a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T13:10:26.693 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:26 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:26.403+0000 7fd66543d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T13:10:26.693 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:26 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:26.500+0000 7fd66543d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T13:10:27.004 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:26 smithi159 ceph-mon[27461]: Standby manager daemon z restarted 2024-02-07T13:10:27.004 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:26 smithi159 ceph-mon[27461]: Standby manager daemon z started 2024-02-07T13:10:27.005 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:26 smithi159 ceph-mon[27461]: from='mgr.? 172.21.15.191:0/2063037975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-07T13:10:27.005 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:26 smithi159 ceph-mon[27461]: from='mgr.? 172.21.15.191:0/2063037975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:10:27.005 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:26 smithi159 ceph-mon[27461]: from='mgr.? 172.21.15.191:0/2063037975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-07T13:10:27.005 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:26 smithi159 ceph-mon[27461]: from='mgr.? 172.21.15.191:0/2063037975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:10:27.005 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:26 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:26.718+0000 7fd66543d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T13:10:27.005 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:26 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: [07/Feb/2024:13:10:26] ENGINE Bus STARTING 2024-02-07T13:10:27.005 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:26 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: CherryPy Checker: 2024-02-07T13:10:27.005 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:26 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: The Application mounted at '' has an empty config. 2024-02-07T13:10:27.005 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:26 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: 2024-02-07T13:10:27.005 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:26 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: [07/Feb/2024:13:10:26] ENGINE Serving on http://:::9283 2024-02-07T13:10:27.005 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:10:26 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x[29151]: [07/Feb/2024:13:10:26] ENGINE Bus STARTED 2024-02-07T13:10:27.026 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:26 smithi067 ceph-mon[25698]: Standby manager daemon z restarted 2024-02-07T13:10:27.026 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:26 smithi067 ceph-mon[25698]: Standby manager daemon z started 2024-02-07T13:10:27.026 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:26 smithi067 ceph-mon[25698]: from='mgr.? 172.21.15.191:0/2063037975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-07T13:10:27.026 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:26 smithi067 ceph-mon[25698]: from='mgr.? 172.21.15.191:0/2063037975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:10:27.026 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:26 smithi067 ceph-mon[25698]: from='mgr.? 172.21.15.191:0/2063037975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-07T13:10:27.026 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:26 smithi067 ceph-mon[25698]: from='mgr.? 172.21.15.191:0/2063037975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:10:27.027 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:26 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:26.659+0000 7ff36233a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-07T13:10:27.027 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:26 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:26.767+0000 7ff36233a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-07T13:10:27.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:26 smithi191 ceph-mon[27623]: Standby manager daemon z restarted 2024-02-07T13:10:27.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:26 smithi191 ceph-mon[27623]: Standby manager daemon z started 2024-02-07T13:10:27.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:26 smithi191 ceph-mon[27623]: from='mgr.? 172.21.15.191:0/2063037975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-07T13:10:27.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:26 smithi191 ceph-mon[27623]: from='mgr.? 172.21.15.191:0/2063037975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:10:27.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:26 smithi191 ceph-mon[27623]: from='mgr.? 172.21.15.191:0/2063037975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-07T13:10:27.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:26 smithi191 ceph-mon[27623]: from='mgr.? 172.21.15.191:0/2063037975' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:10:27.120 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:26 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:26.659+0000 7f0ad848a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T13:10:27.120 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:26 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: [07/Feb/2024:13:10:26] ENGINE Bus STARTING 2024-02-07T13:10:27.120 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:26 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: CherryPy Checker: 2024-02-07T13:10:27.120 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:26 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: The Application mounted at '' has an empty config. 2024-02-07T13:10:27.120 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:26 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: 2024-02-07T13:10:27.120 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:26 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: [07/Feb/2024:13:10:26] ENGINE Serving on http://:::9283 2024-02-07T13:10:27.120 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:10:26 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z[30077]: [07/Feb/2024:13:10:26] ENGINE Bus STARTED 2024-02-07T13:10:27.278 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:27.027+0000 7ff36233a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-07T13:10:27.279 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:10:27] ENGINE Bus STARTING 2024-02-07T13:10:27.279 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: CherryPy Checker: 2024-02-07T13:10:27.279 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: The Application mounted at '' has an empty config. 2024-02-07T13:10:27.279 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:10:27.279 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:10:27] ENGINE Serving on http://:::9283 2024-02-07T13:10:27.555 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:10:27] ENGINE Bus STARTED 2024-02-07T13:10:27.884 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: mgrmap e18: y(active, since 3m), standbys: x, z 2024-02-07T13:10:27.884 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: Standby manager daemon x restarted 2024-02-07T13:10:27.884 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: Standby manager daemon x started 2024-02-07T13:10:27.884 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.? 172.21.15.159:0/1233424204' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-07T13:10:27.884 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.? 172.21.15.159:0/1233424204' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:10:27.884 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.? 172.21.15.159:0/1233424204' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-07T13:10:27.884 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.? 172.21.15.159:0/1233424204' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:10:27.884 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: Active manager daemon y restarted 2024-02-07T13:10:27.884 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: Activating manager daemon y 2024-02-07T13:10:27.884 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: osdmap e32: 3 total, 3 up, 3 in 2024-02-07T13:10:27.884 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: mgrmap e19: y(active, starting, since 0.0105029s), standbys: x, z 2024-02-07T13:10:27.885 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:10:27.885 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:10:27.885 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:10:27.885 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-07T13:10:27.885 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-07T13:10:27.885 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-07T13:10:27.885 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:10:27.885 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:10:27.885 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:10:27.885 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-07T13:10:27.885 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-07T13:10:27.885 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-07T13:10:27.885 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: Manager daemon y is now available 2024-02-07T13:10:27.885 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:27.886 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:10:27.886 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:10:27.886 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:10:27.886 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]': finished 2024-02-07T13:10:27.886 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi191", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:10:27.886 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:10:27.886 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi191", "name": "osd_memory_target"}]': finished 2024-02-07T13:10:27.886 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:27.886 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"}]': finished 2024-02-07T13:10:27.886 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:10:27.886 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:10:27.886 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:10:27.886 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-07T13:10:27.886 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-07T13:10:27.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: mgrmap e18: y(active, since 3m), standbys: x, z 2024-02-07T13:10:27.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: Standby manager daemon x restarted 2024-02-07T13:10:27.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: Standby manager daemon x started 2024-02-07T13:10:27.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.? 172.21.15.159:0/1233424204' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-07T13:10:27.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.? 172.21.15.159:0/1233424204' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:10:27.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.? 172.21.15.159:0/1233424204' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-07T13:10:27.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.? 172.21.15.159:0/1233424204' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:10:27.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: Active manager daemon y restarted 2024-02-07T13:10:27.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: Activating manager daemon y 2024-02-07T13:10:27.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: osdmap e32: 3 total, 3 up, 3 in 2024-02-07T13:10:27.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: mgrmap e19: y(active, starting, since 0.0105029s), standbys: x, z 2024-02-07T13:10:27.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:10:27.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:10:27.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:10:27.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-07T13:10:27.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-07T13:10:27.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-07T13:10:27.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:10:27.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:10:27.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:10:27.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-07T13:10:27.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-07T13:10:27.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-07T13:10:27.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: Manager daemon y is now available 2024-02-07T13:10:27.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:27.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:10:27.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:10:27.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:10:27.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]': finished 2024-02-07T13:10:27.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi191", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:10:27.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:10:27.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi191", "name": "osd_memory_target"}]': finished 2024-02-07T13:10:27.945 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:27.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"}]': finished 2024-02-07T13:10:27.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:10:27.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:10:27.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:10:27.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-07T13:10:27.946 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-07T13:10:28.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: mgrmap e18: y(active, since 3m), standbys: x, z 2024-02-07T13:10:28.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: Standby manager daemon x restarted 2024-02-07T13:10:28.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: Standby manager daemon x started 2024-02-07T13:10:28.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.? 172.21.15.159:0/1233424204' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-07T13:10:28.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.? 172.21.15.159:0/1233424204' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-07T13:10:28.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.? 172.21.15.159:0/1233424204' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-07T13:10:28.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.? 172.21.15.159:0/1233424204' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-07T13:10:28.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: Active manager daemon y restarted 2024-02-07T13:10:28.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: Activating manager daemon y 2024-02-07T13:10:28.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: osdmap e32: 3 total, 3 up, 3 in 2024-02-07T13:10:28.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: mgrmap e19: y(active, starting, since 0.0105029s), standbys: x, z 2024-02-07T13:10:28.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-07T13:10:28.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-07T13:10:28.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-07T13:10:28.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-07T13:10:28.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-07T13:10:28.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-07T13:10:28.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-07T13:10:28.121 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-07T13:10:28.121 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-07T13:10:28.121 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-07T13:10:28.121 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-07T13:10:28.121 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-07T13:10:28.121 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: Manager daemon y is now available 2024-02-07T13:10:28.121 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:28.121 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:10:28.121 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:10:28.121 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:10:28.121 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi159", "name": "osd_memory_target"}]': finished 2024-02-07T13:10:28.121 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi191", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:10:28.121 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"}]: dispatch 2024-02-07T13:10:28.121 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi191", "name": "osd_memory_target"}]': finished 2024-02-07T13:10:28.122 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:28.122 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"}]': finished 2024-02-07T13:10:28.122 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:10:28.122 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-07T13:10:28.122 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-07T13:10:28.122 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-07T13:10:28.122 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-07T13:10:28.224 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:10:28.224 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":32,"fsid":"81df3e5a-c5b9-11ee-95b6-87774f69a715","created":"2024-02-07T13:06:11.589197+0000","modified":"2024-02-07T13:10:27.040755+0000","last_up_change":"2024-02-07T13:09:43.655215+0000","last_in_change":"2024-02-07T13:09:22.920752+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-07T13:09:45.526400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-07T13:09:55.860713+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-02-07T13:09:57.358100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-02-07T13:09:59.320263+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"29","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-02-07T13:10:01.371824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e53b131-f387-4600-b27c-e421ff58b822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":4131814377},{"type":"v1","addr":"172.21.15.67:6803","nonce":4131814377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":4131814377},{"type":"v1","addr":"172.21.15.67:6805","nonce":4131814377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":4131814377},{"type":"v1","addr":"172.21.15.67:6809","nonce":4131814377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":4131814377},{"type":"v1","addr":"172.21.15.67:6807","nonce":4131814377}]},"public_addr":"172.21.15.67:6803/4131814377","cluster_addr":"172.21.15.67:6805/4131814377","heartbeat_back_addr":"172.21.15.67:6809/4131814377","heartbeat_front_addr":"172.21.15.67:6807/4131814377","state":["exists","up"]},{"osd":1,"uuid":"24892153-adea-4764-b22f-de70793f3fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6800","nonce":80909837},{"type":"v1","addr":"172.21.15.159:6801","nonce":80909837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6802","nonce":80909837},{"type":"v1","addr":"172.21.15.159:6803","nonce":80909837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6806","nonce":80909837},{"type":"v1","addr":"172.21.15.159:6807","nonce":80909837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.159:6804","nonce":80909837},{"type":"v1","addr":"172.21.15.159:6805","nonce":80909837}]},"public_addr":"172.21.15.159:6801/80909837","cluster_addr":"172.21.15.159:6803/80909837","heartbeat_back_addr":"172.21.15.159:6807/80909837","heartbeat_front_addr":"172.21.15.159:6805/80909837","state":["exists","up"]},{"osd":2,"uuid":"6fac69df-3cda-49c6-b775-2822c1e58238","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6800","nonce":2149484224},{"type":"v1","addr":"172.21.15.191:6801","nonce":2149484224}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6802","nonce":2149484224},{"type":"v1","addr":"172.21.15.191:6803","nonce":2149484224}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6806","nonce":2149484224},{"type":"v1","addr":"172.21.15.191:6807","nonce":2149484224}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.191:6804","nonce":2149484224},{"type":"v1","addr":"172.21.15.191:6805","nonce":2149484224}]},"public_addr":"172.21.15.191:6801/2149484224","cluster_addr":"172.21.15.191:6803/2149484224","heartbeat_back_addr":"172.21.15.191:6807/2149484224","heartbeat_front_addr":"172.21.15.191:6805/2149484224","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T13:08:48.327715+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T13:09:14.092634+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701272860131327,"old_weight":0,"last_purged_snaps_scrub":"2024-02-07T13:09:41.551859+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.67:0/844469757":"2024-02-08T13:10:27.040725+0000","172.21.15.67:0/3739908062":"2024-02-08T13:10:27.040725+0000","172.21.15.67:0/3548590016":"2024-02-08T13:10:27.040725+0000","172.21.15.67:6801/3886435222":"2024-02-08T13:10:27.040725+0000","172.21.15.67:6800/3886435222":"2024-02-08T13:10:27.040725+0000","172.21.15.67:0/416625014":"2024-02-08T13:07:09.178941+0000","172.21.15.67:6801/1299844367":"2024-02-08T13:07:09.178941+0000","172.21.15.67:6800/1299844367":"2024-02-08T13:07:09.178941+0000","172.21.15.67:0/4116703611":"2024-02-08T13:07:09.178941+0000","172.21.15.67:6800/3665964957":"2024-02-08T13:06:41.781073+0000","172.21.15.67:6801/3665964957":"2024-02-08T13:06:41.781073+0000","172.21.15.67:0/3779372659":"2024-02-08T13:10:27.040725+0000","172.21.15.67:0/2797980579":"2024-02-08T13:06:41.781073+0000","172.21.15.67:0/3554625152":"2024-02-08T13:06:41.781073+0000","172.21.15.67:0/3615206419":"2024-02-08T13:06:41.781073+0000","172.21.15.67:0/3516969484":"2024-02-08T13:07:09.178941+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-07T13:10:28.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:28 smithi067 ceph-mon[25698]: Updating smithi067:/etc/ceph/ceph.conf 2024-02-07T13:10:28.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:28 smithi067 ceph-mon[25698]: Updating smithi159:/etc/ceph/ceph.conf 2024-02-07T13:10:28.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:28 smithi067 ceph-mon[25698]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-07T13:10:28.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:28 smithi067 ceph-mon[25698]: Updating smithi067:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:10:28.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:28 smithi067 ceph-mon[25698]: Updating smithi159:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:10:28.871 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:28 smithi067 ceph-mon[25698]: Updating smithi191:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:10:28.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:28 smithi067 ceph-mon[25698]: mgrmap e20: y(active, since 1.01892s), standbys: x, z 2024-02-07T13:10:28.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:28 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/628763633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T13:10:28.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:28 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:28.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:28 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:28.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:28 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:28.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:28 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:28.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:28 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:28.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:28 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:28.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:28 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:28.872 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:28 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:28.899 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-07T13:10:28.900 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-07T13:10:28.900 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-07T13:10:28.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:28 smithi159 ceph-mon[27461]: Updating smithi067:/etc/ceph/ceph.conf 2024-02-07T13:10:28.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:28 smithi159 ceph-mon[27461]: Updating smithi159:/etc/ceph/ceph.conf 2024-02-07T13:10:28.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:28 smithi159 ceph-mon[27461]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-07T13:10:28.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:28 smithi159 ceph-mon[27461]: Updating smithi067:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:10:28.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:28 smithi159 ceph-mon[27461]: Updating smithi159:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:10:28.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:28 smithi159 ceph-mon[27461]: Updating smithi191:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:10:28.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:28 smithi159 ceph-mon[27461]: mgrmap e20: y(active, since 1.01892s), standbys: x, z 2024-02-07T13:10:28.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:28 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/628763633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T13:10:28.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:28 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:28.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:28 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:28.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:28 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:28.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:28 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:28.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:28 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:28.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:28 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:28.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:28 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:28.944 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:28 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:29.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:28 smithi191 ceph-mon[27623]: Updating smithi067:/etc/ceph/ceph.conf 2024-02-07T13:10:29.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:28 smithi191 ceph-mon[27623]: Updating smithi159:/etc/ceph/ceph.conf 2024-02-07T13:10:29.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:28 smithi191 ceph-mon[27623]: Updating smithi191:/etc/ceph/ceph.conf 2024-02-07T13:10:29.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:28 smithi191 ceph-mon[27623]: Updating smithi067:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:10:29.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:28 smithi191 ceph-mon[27623]: Updating smithi159:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:10:29.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:28 smithi191 ceph-mon[27623]: Updating smithi191:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.conf 2024-02-07T13:10:29.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:28 smithi191 ceph-mon[27623]: mgrmap e20: y(active, since 1.01892s), standbys: x, z 2024-02-07T13:10:29.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:28 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/628763633' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-07T13:10:29.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:28 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:29.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:28 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:29.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:28 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:29.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:28 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:29.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:28 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:29.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:28 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:29.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:28 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:29.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:28 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:29.188 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:10:29.189 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:10:29.191 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:10:30.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:29 smithi191 ceph-mon[27623]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:30.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:29 smithi191 ceph-mon[27623]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:10:30.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:29 smithi191 ceph-mon[27623]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:10:30.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:29 smithi191 ceph-mon[27623]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:10:30.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:29 smithi191 ceph-mon[27623]: [07/Feb/2024:13:10:28] ENGINE Bus STARTING 2024-02-07T13:10:30.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:29 smithi191 ceph-mon[27623]: Updating smithi067:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:10:30.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:29 smithi191 ceph-mon[27623]: Updating smithi191:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:10:30.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:29 smithi191 ceph-mon[27623]: Updating smithi159:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:10:30.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:29 smithi191 ceph-mon[27623]: [07/Feb/2024:13:10:28] ENGINE Serving on https://172.21.15.67:7151 2024-02-07T13:10:30.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:29 smithi191 ceph-mon[27623]: [07/Feb/2024:13:10:28] ENGINE Serving on http://172.21.15.67:8765 2024-02-07T13:10:30.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:29 smithi191 ceph-mon[27623]: [07/Feb/2024:13:10:28] ENGINE Bus STARTED 2024-02-07T13:10:30.120 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:29 smithi191 ceph-mon[27623]: Deploying daemon alertmanager.a on smithi067 2024-02-07T13:10:30.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:29 smithi159 ceph-mon[27461]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:30.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:29 smithi159 ceph-mon[27461]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:10:30.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:29 smithi159 ceph-mon[27461]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:10:30.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:29 smithi159 ceph-mon[27461]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:10:30.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:29 smithi159 ceph-mon[27461]: [07/Feb/2024:13:10:28] ENGINE Bus STARTING 2024-02-07T13:10:30.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:29 smithi159 ceph-mon[27461]: Updating smithi067:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:10:30.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:29 smithi159 ceph-mon[27461]: Updating smithi191:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:10:30.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:29 smithi159 ceph-mon[27461]: Updating smithi159:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:10:30.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:29 smithi159 ceph-mon[27461]: [07/Feb/2024:13:10:28] ENGINE Serving on https://172.21.15.67:7151 2024-02-07T13:10:30.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:29 smithi159 ceph-mon[27461]: [07/Feb/2024:13:10:28] ENGINE Serving on http://172.21.15.67:8765 2024-02-07T13:10:30.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:29 smithi159 ceph-mon[27461]: [07/Feb/2024:13:10:28] ENGINE Bus STARTED 2024-02-07T13:10:30.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:29 smithi159 ceph-mon[27461]: Deploying daemon alertmanager.a on smithi067 2024-02-07T13:10:30.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:29 smithi067 ceph-mon[25698]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:30.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:29 smithi067 ceph-mon[25698]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:10:30.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:29 smithi067 ceph-mon[25698]: Updating smithi191:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:10:30.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:29 smithi067 ceph-mon[25698]: Updating smithi159:/etc/ceph/ceph.client.admin.keyring 2024-02-07T13:10:30.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:29 smithi067 ceph-mon[25698]: [07/Feb/2024:13:10:28] ENGINE Bus STARTING 2024-02-07T13:10:30.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:29 smithi067 ceph-mon[25698]: Updating smithi067:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:10:30.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:29 smithi067 ceph-mon[25698]: Updating smithi191:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:10:30.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:29 smithi067 ceph-mon[25698]: Updating smithi159:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/config/ceph.client.admin.keyring 2024-02-07T13:10:30.246 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:29 smithi067 ceph-mon[25698]: [07/Feb/2024:13:10:28] ENGINE Serving on https://172.21.15.67:7151 2024-02-07T13:10:30.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:29 smithi067 ceph-mon[25698]: [07/Feb/2024:13:10:28] ENGINE Serving on http://172.21.15.67:8765 2024-02-07T13:10:30.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:29 smithi067 ceph-mon[25698]: [07/Feb/2024:13:10:28] ENGINE Bus STARTED 2024-02-07T13:10:30.247 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:29 smithi067 ceph-mon[25698]: Deploying daemon alertmanager.a on smithi067 2024-02-07T13:10:31.040 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:30 smithi191 ceph-mon[27623]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:31.041 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:30 smithi191 ceph-mon[27623]: mgrmap e21: y(active, since 3s), standbys: x, z 2024-02-07T13:10:31.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:30 smithi159 ceph-mon[27461]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:31.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:30 smithi159 ceph-mon[27461]: mgrmap e21: y(active, since 3s), standbys: x, z 2024-02-07T13:10:31.241 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:30 smithi067 ceph-mon[25698]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:31.241 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:30 smithi067 ceph-mon[25698]: mgrmap e21: y(active, since 3s), standbys: x, z 2024-02-07T13:10:31.555 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:31 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: ::ffff:172.21.15.159 - - [07/Feb/2024:13:10:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-07T13:10:33.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:32 smithi191 ceph-mon[27623]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:33.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:32 smithi191 ceph-mon[27623]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-02-07T13:10:33.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:32 smithi159 ceph-mon[27461]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:33.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:32 smithi159 ceph-mon[27461]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-02-07T13:10:33.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:32 smithi067 ceph-mon[25698]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:33.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:32 smithi067 ceph-mon[25698]: mgrmap e22: y(active, since 4s), standbys: x, z 2024-02-07T13:10:34.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:34 smithi159 ceph-mon[27461]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:35.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:34 smithi067 ceph-mon[25698]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:35.118 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:34 smithi191 ceph-mon[27623]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:36.460 INFO:teuthology.orchestra.run.smithi067.stdout:34359738391 2024-02-07T13:10:36.461 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-07T13:10:36.904 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:36 smithi159 ceph-mon[27461]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-07T13:10:37.158 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:36 smithi067 ceph-mon[25698]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-07T13:10:37.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:36 smithi191 ceph-mon[27623]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-07T13:10:38.390 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:10:39.123 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:38 smithi067 ceph-mon[25698]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-07T13:10:39.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:38 smithi159 ceph-mon[27461]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-07T13:10:39.368 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:38 smithi191 ceph-mon[27623]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-07T13:10:39.948 INFO:teuthology.orchestra.run.smithi067.stdout:55834574866 2024-02-07T13:10:39.948 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-07T13:10:40.900 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:40 smithi067 ceph-mon[25698]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-07T13:10:41.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:40 smithi159 ceph-mon[27461]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-07T13:10:41.368 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:40 smithi191 ceph-mon[27623]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-07T13:10:41.412 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:41 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: ::ffff:172.21.15.159 - - [07/Feb/2024:13:10:41] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-07T13:10:41.413 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:10:41 smithi067 systemd[1]: Starting Ceph alertmanager.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:10:41.805 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:10:42.209 INFO:teuthology.orchestra.run.smithi067.stdout:77309411342 2024-02-07T13:10:42.209 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-07T13:10:42.805 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:10:42 smithi067 podman[47629]: 2024-02-07 13:10:42.376919391 +0000 UTC m=+0.105357148 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-02-07T13:10:42.806 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:10:42 smithi067 podman[47629]: 2024-02-07 13:10:42.595637074 +0000 UTC m=+0.324074876 volume create 15101baa307ef5aa9d6c77b476594c24a3308adb3f3754062f23649a2c5b9ed4 2024-02-07T13:10:42.806 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:10:42 smithi067 podman[47629]: 2024-02-07 13:10:42.713452866 +0000 UTC m=+0.441890661 container create 017b9537c5bc3a4fd6a7c852f9551812312bbf65141c13b85d21f13570d4c359 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-07T13:10:43.159 INFO:teuthology.orchestra.run.smithi067.stdout:34359738392 2024-02-07T13:10:43.159 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:42 smithi067 ceph-mon[25698]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T13:10:43.159 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:42 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:10:43.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:42 smithi159 ceph-mon[27461]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T13:10:43.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:42 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:10:43.368 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:42 smithi191 ceph-mon[27623]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T13:10:43.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:42 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:10:44.174 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:43 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/2877581658' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-07T13:10:44.175 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:10:43 smithi067 podman[47629]: 2024-02-07 13:10:43.879764409 +0000 UTC m=+1.608202166 container init 017b9537c5bc3a4fd6a7c852f9551812312bbf65141c13b85d21f13570d4c359 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-07T13:10:44.175 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:10:43 smithi067 podman[47629]: 2024-02-07 13:10:43.887117935 +0000 UTC m=+1.615555695 container start 017b9537c5bc3a4fd6a7c852f9551812312bbf65141c13b85d21f13570d4c359 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-07T13:10:44.175 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:10:43 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-alertmanager-a[47736]: ts=2024-02-07T13:10:43.900Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-07T13:10:44.175 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:10:43 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-alertmanager-a[47736]: ts=2024-02-07T13:10:43.900Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-07T13:10:44.175 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:10:43 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-alertmanager-a[47736]: ts=2024-02-07T13:10:43.901Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.67 port=9094 2024-02-07T13:10:44.175 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:10:43 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-alertmanager-a[47736]: ts=2024-02-07T13:10:43.902Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-07T13:10:44.175 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:10:43 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-alertmanager-a[47736]: ts=2024-02-07T13:10:43.925Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-07T13:10:44.175 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:10:43 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-alertmanager-a[47736]: ts=2024-02-07T13:10:43.926Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-07T13:10:44.175 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:10:43 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-alertmanager-a[47736]: ts=2024-02-07T13:10:43.929Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-07T13:10:44.176 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:10:43 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-alertmanager-a[47736]: ts=2024-02-07T13:10:43.929Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-07T13:10:44.176 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:10:43 smithi067 bash[47629]: 017b9537c5bc3a4fd6a7c852f9551812312bbf65141c13b85d21f13570d4c359 2024-02-07T13:10:44.176 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:10:43 smithi067 systemd[1]: Started Ceph alertmanager.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:10:44.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:43 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/2877581658' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-07T13:10:44.368 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:43 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/2877581658' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-07T13:10:44.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:44 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:10:44] ENGINE Bus STOPPING 2024-02-07T13:10:44.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:44 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:10:44] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-07T13:10:44.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:44 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:10:44] ENGINE Bus STOPPED 2024-02-07T13:10:44.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:44 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:10:44] ENGINE Bus STARTING 2024-02-07T13:10:44.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:44 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:10:44] ENGINE Serving on http://:::9283 2024-02-07T13:10:44.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:44 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:10:44] ENGINE Bus STARTED 2024-02-07T13:10:44.899 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:10:44.927 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738391 got 34359738392 for osd.0 2024-02-07T13:10:44.927 DEBUG:teuthology.parallel:result is None 2024-02-07T13:10:45.169 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:44 smithi067 ceph-mon[25698]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T13:10:45.169 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:44 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:44 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:44 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:44 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:44 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:44 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:44 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-07T13:10:45.170 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:44 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:44 smithi159 ceph-mon[27461]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T13:10:45.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:44 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:44 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:44 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:44 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:44 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:44 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:44 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-07T13:10:45.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:44 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:44 smithi191 ceph-mon[27623]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T13:10:45.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:44 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:44 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:44 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:44 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:44 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:44 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:45.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:44 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-07T13:10:45.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:44 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:46.055 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:10:45 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-alertmanager-a[47736]: ts=2024-02-07T13:10:45.902Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000143093s 2024-02-07T13:10:46.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:45 smithi067 ceph-mon[25698]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-07T13:10:46.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:45 smithi067 ceph-mon[25698]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-07T13:10:46.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:45 smithi067 ceph-mon[25698]: Deploying daemon grafana.a on smithi159 2024-02-07T13:10:46.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:45 smithi159 ceph-mon[27461]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-07T13:10:46.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:45 smithi159 ceph-mon[27461]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-07T13:10:46.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:45 smithi159 ceph-mon[27461]: Deploying daemon grafana.a on smithi159 2024-02-07T13:10:46.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:45 smithi191 ceph-mon[27623]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-07T13:10:46.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:45 smithi191 ceph-mon[27623]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-07T13:10:46.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:45 smithi191 ceph-mon[27623]: Deploying daemon grafana.a on smithi159 2024-02-07T13:10:47.001 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:46 smithi067 ceph-mon[25698]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T13:10:47.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:46 smithi159 ceph-mon[27461]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T13:10:47.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:46 smithi191 ceph-mon[27623]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-07T13:10:47.947 INFO:teuthology.orchestra.run.smithi067.stdout:55834574868 2024-02-07T13:10:48.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:48 smithi067 ceph-mon[25698]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:48.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:48 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:48.300 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:48 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/3101455831' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-07T13:10:48.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:48 smithi159 ceph-mon[27461]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:48.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:48 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:48.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:48 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/3101455831' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-07T13:10:48.585 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574866 got 55834574868 for osd.1 2024-02-07T13:10:48.585 DEBUG:teuthology.parallel:result is None 2024-02-07T13:10:48.618 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:48 smithi191 ceph-mon[27623]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:48.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:48 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:10:48.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:48 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/3101455831' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-07T13:10:49.006 INFO:teuthology.orchestra.run.smithi067.stdout:77309411343 2024-02-07T13:10:50.082 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:49 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/3669922850' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-07T13:10:50.108 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411342 got 77309411343 for osd.2 2024-02-07T13:10:50.108 DEBUG:teuthology.parallel:result is None 2024-02-07T13:10:50.108 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-07T13:10:50.108 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph pg dump --format=json 2024-02-07T13:10:50.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:49 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/3669922850' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-07T13:10:50.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:49 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/3669922850' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-07T13:10:51.050 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:50 smithi067 ceph-mon[25698]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:51.058 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:50 smithi191 ceph-mon[27623]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:51.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:50 smithi159 ceph-mon[27461]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:51.305 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:10:51 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: ::ffff:172.21.15.159 - - [07/Feb/2024:13:10:51] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-07T13:10:51.511 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:10:53.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:52 smithi067 ceph-mon[25698]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:53.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:52 smithi191 ceph-mon[27623]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:53.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:52 smithi159 ceph-mon[27461]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:54.306 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:10:53 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-alertmanager-a[47736]: ts=2024-02-07T13:10:53.904Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.001822914s 2024-02-07T13:10:55.052 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:54 smithi067 ceph-mon[25698]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:55.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:54 smithi159 ceph-mon[27461]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:55.368 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:54 smithi191 ceph-mon[27623]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:55.509 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:10:55.511 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-07T13:10:56.202 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-02-07T13:10:55.063697+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595908,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":974,"num_read_kb":821,"num_write":697,"num_write_kb":4546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":673,"ondisk_log_size":673,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":96640,"kb_used_data":3108,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":281126528,"statfs":{"total":287972524032,"available":287873564672,"internally_reserved":0,"allocated":3182592,"data_stored":2109330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002596"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047340+0000","last_change":"2024-02-07T13:10:03.088209+0000","last_active":"2024-02-07T13:10:27.047340+0000","last_peered":"2024-02-07T13:10:27.047340+0000","last_clean":"2024-02-07T13:10:27.047340+0000","last_became_active":"2024-02-07T13:10:03.087867+0000","last_became_peered":"2024-02-07T13:10:03.087867+0000","last_unstale":"2024-02-07T13:10:27.047340+0000","last_undegraded":"2024-02-07T13:10:27.047340+0000","last_fullsized":"2024-02-07T13:10:27.047340+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:27:05.993847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047025+0000","last_change":"2024-02-07T13:10:01.069894+0000","last_active":"2024-02-07T13:10:27.047025+0000","last_peered":"2024-02-07T13:10:27.047025+0000","last_clean":"2024-02-07T13:10:27.047025+0000","last_became_active":"2024-02-07T13:10:01.069609+0000","last_became_peered":"2024-02-07T13:10:01.069609+0000","last_unstale":"2024-02-07T13:10:27.047025+0000","last_undegraded":"2024-02-07T13:10:27.047025+0000","last_fullsized":"2024-02-07T13:10:27.047025+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:56:48.582558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049521+0000","last_change":"2024-02-07T13:09:57.046326+0000","last_active":"2024-02-07T13:10:27.049521+0000","last_peered":"2024-02-07T13:10:27.049521+0000","last_clean":"2024-02-07T13:10:27.049521+0000","last_became_active":"2024-02-07T13:09:57.045877+0000","last_became_peered":"2024-02-07T13:09:57.045877+0000","last_unstale":"2024-02-07T13:10:27.049521+0000","last_undegraded":"2024-02-07T13:10:27.049521+0000","last_fullsized":"2024-02-07T13:10:27.049521+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:07:16.828704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"31'10","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049402+0000","last_change":"2024-02-07T13:09:59.060952+0000","last_active":"2024-02-07T13:10:27.049402+0000","last_peered":"2024-02-07T13:10:27.049402+0000","last_clean":"2024-02-07T13:10:27.049402+0000","last_became_active":"2024-02-07T13:09:59.060387+0000","last_became_peered":"2024-02-07T13:09:59.060387+0000","last_unstale":"2024-02-07T13:10:27.049402+0000","last_undegraded":"2024-02-07T13:10:27.049402+0000","last_fullsized":"2024-02-07T13:10:27.049402+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:14:57.814494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046053+0000","last_change":"2024-02-07T13:10:03.083060+0000","last_active":"2024-02-07T13:10:27.046053+0000","last_peered":"2024-02-07T13:10:27.046053+0000","last_clean":"2024-02-07T13:10:27.046053+0000","last_became_active":"2024-02-07T13:10:03.082572+0000","last_became_peered":"2024-02-07T13:10:03.082572+0000","last_unstale":"2024-02-07T13:10:27.046053+0000","last_undegraded":"2024-02-07T13:10:27.046053+0000","last_fullsized":"2024-02-07T13:10:27.046053+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:40:35.919055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047189+0000","last_change":"2024-02-07T13:10:01.072342+0000","last_active":"2024-02-07T13:10:27.047189+0000","last_peered":"2024-02-07T13:10:27.047189+0000","last_clean":"2024-02-07T13:10:27.047189+0000","last_became_active":"2024-02-07T13:10:01.071845+0000","last_became_peered":"2024-02-07T13:10:01.071845+0000","last_unstale":"2024-02-07T13:10:27.047189+0000","last_undegraded":"2024-02-07T13:10:27.047189+0000","last_fullsized":"2024-02-07T13:10:27.047189+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:24:00.754390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049566+0000","last_change":"2024-02-07T13:09:57.046796+0000","last_active":"2024-02-07T13:10:27.049566+0000","last_peered":"2024-02-07T13:10:27.049566+0000","last_clean":"2024-02-07T13:10:27.049566+0000","last_became_active":"2024-02-07T13:09:57.046159+0000","last_became_peered":"2024-02-07T13:09:57.046159+0000","last_unstale":"2024-02-07T13:10:27.049566+0000","last_undegraded":"2024-02-07T13:10:27.049566+0000","last_fullsized":"2024-02-07T13:10:27.049566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:08:29.765095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"31'11","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049177+0000","last_change":"2024-02-07T13:09:59.060218+0000","last_active":"2024-02-07T13:10:27.049177+0000","last_peered":"2024-02-07T13:10:27.049177+0000","last_clean":"2024-02-07T13:10:27.049177+0000","last_became_active":"2024-02-07T13:09:59.059734+0000","last_became_peered":"2024-02-07T13:09:59.059734+0000","last_unstale":"2024-02-07T13:10:27.049177+0000","last_undegraded":"2024-02-07T13:10:27.049177+0000","last_fullsized":"2024-02-07T13:10:27.049177+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:42:27.742686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047155+0000","last_change":"2024-02-07T13:10:03.084542+0000","last_active":"2024-02-07T13:10:27.047155+0000","last_peered":"2024-02-07T13:10:27.047155+0000","last_clean":"2024-02-07T13:10:27.047155+0000","last_became_active":"2024-02-07T13:10:03.084107+0000","last_became_peered":"2024-02-07T13:10:03.084107+0000","last_unstale":"2024-02-07T13:10:27.047155+0000","last_undegraded":"2024-02-07T13:10:27.047155+0000","last_fullsized":"2024-02-07T13:10:27.047155+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:34:52.337365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048073+0000","last_change":"2024-02-07T13:10:01.072094+0000","last_active":"2024-02-07T13:10:27.048073+0000","last_peered":"2024-02-07T13:10:27.048073+0000","last_clean":"2024-02-07T13:10:27.048073+0000","last_became_active":"2024-02-07T13:10:01.071629+0000","last_became_peered":"2024-02-07T13:10:01.071629+0000","last_unstale":"2024-02-07T13:10:27.048073+0000","last_undegraded":"2024-02-07T13:10:27.048073+0000","last_fullsized":"2024-02-07T13:10:27.048073+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:47:39.945764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049620+0000","last_change":"2024-02-07T13:09:57.050009+0000","last_active":"2024-02-07T13:10:27.049620+0000","last_peered":"2024-02-07T13:10:27.049620+0000","last_clean":"2024-02-07T13:10:27.049620+0000","last_became_active":"2024-02-07T13:09:57.048095+0000","last_became_peered":"2024-02-07T13:09:57.048095+0000","last_unstale":"2024-02-07T13:10:27.049620+0000","last_undegraded":"2024-02-07T13:10:27.049620+0000","last_fullsized":"2024-02-07T13:10:27.049620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:30:07.938219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"31'15","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049081+0000","last_change":"2024-02-07T13:09:59.057441+0000","last_active":"2024-02-07T13:10:27.049081+0000","last_peered":"2024-02-07T13:10:27.049081+0000","last_clean":"2024-02-07T13:10:27.049081+0000","last_became_active":"2024-02-07T13:09:59.057080+0000","last_became_peered":"2024-02-07T13:09:59.057080+0000","last_unstale":"2024-02-07T13:10:27.049081+0000","last_undegraded":"2024-02-07T13:10:27.049081+0000","last_fullsized":"2024-02-07T13:10:27.049081+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:24:12.244346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047800+0000","last_change":"2024-02-07T13:10:03.085119+0000","last_active":"2024-02-07T13:10:27.047800+0000","last_peered":"2024-02-07T13:10:27.047800+0000","last_clean":"2024-02-07T13:10:27.047800+0000","last_became_active":"2024-02-07T13:10:03.084349+0000","last_became_peered":"2024-02-07T13:10:03.084349+0000","last_unstale":"2024-02-07T13:10:27.047800+0000","last_undegraded":"2024-02-07T13:10:27.047800+0000","last_fullsized":"2024-02-07T13:10:27.047800+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:36:57.272469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047394+0000","last_change":"2024-02-07T13:10:01.070229+0000","last_active":"2024-02-07T13:10:27.047394+0000","last_peered":"2024-02-07T13:10:27.047394+0000","last_clean":"2024-02-07T13:10:27.047394+0000","last_became_active":"2024-02-07T13:10:01.069778+0000","last_became_peered":"2024-02-07T13:10:01.069778+0000","last_unstale":"2024-02-07T13:10:27.047394+0000","last_undegraded":"2024-02-07T13:10:27.047394+0000","last_fullsized":"2024-02-07T13:10:27.047394+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:24:16.008928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048388+0000","last_change":"2024-02-07T13:09:57.049301+0000","last_active":"2024-02-07T13:10:27.048388+0000","last_peered":"2024-02-07T13:10:27.048388+0000","last_clean":"2024-02-07T13:10:27.048388+0000","last_became_active":"2024-02-07T13:09:57.047789+0000","last_became_peered":"2024-02-07T13:09:57.047789+0000","last_unstale":"2024-02-07T13:10:27.048388+0000","last_undegraded":"2024-02-07T13:10:27.048388+0000","last_fullsized":"2024-02-07T13:10:27.048388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:58:25.388168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"31'12","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049553+0000","last_change":"2024-02-07T13:09:59.057769+0000","last_active":"2024-02-07T13:10:27.049553+0000","last_peered":"2024-02-07T13:10:27.049553+0000","last_clean":"2024-02-07T13:10:27.049553+0000","last_became_active":"2024-02-07T13:09:59.057482+0000","last_became_peered":"2024-02-07T13:09:59.057482+0000","last_unstale":"2024-02-07T13:10:27.049553+0000","last_undegraded":"2024-02-07T13:10:27.049553+0000","last_fullsized":"2024-02-07T13:10:27.049553+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:34:29.980756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048643+0000","last_change":"2024-02-07T13:10:03.084807+0000","last_active":"2024-02-07T13:10:27.048643+0000","last_peered":"2024-02-07T13:10:27.048643+0000","last_clean":"2024-02-07T13:10:27.048643+0000","last_became_active":"2024-02-07T13:10:03.084381+0000","last_became_peered":"2024-02-07T13:10:03.084381+0000","last_unstale":"2024-02-07T13:10:27.048643+0000","last_undegraded":"2024-02-07T13:10:27.048643+0000","last_fullsized":"2024-02-07T13:10:27.048643+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:53:40.147298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048615+0000","last_change":"2024-02-07T13:10:01.067690+0000","last_active":"2024-02-07T13:10:27.048615+0000","last_peered":"2024-02-07T13:10:27.048615+0000","last_clean":"2024-02-07T13:10:27.048615+0000","last_became_active":"2024-02-07T13:10:01.067409+0000","last_became_peered":"2024-02-07T13:10:01.067409+0000","last_unstale":"2024-02-07T13:10:27.048615+0000","last_undegraded":"2024-02-07T13:10:27.048615+0000","last_fullsized":"2024-02-07T13:10:27.048615+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:38:49.161734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048732+0000","last_change":"2024-02-07T13:09:57.047606+0000","last_active":"2024-02-07T13:10:27.048732+0000","last_peered":"2024-02-07T13:10:27.048732+0000","last_clean":"2024-02-07T13:10:27.048732+0000","last_became_active":"2024-02-07T13:09:57.046293+0000","last_became_peered":"2024-02-07T13:09:57.046293+0000","last_unstale":"2024-02-07T13:10:27.048732+0000","last_undegraded":"2024-02-07T13:10:27.048732+0000","last_fullsized":"2024-02-07T13:10:27.048732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:54:57.181133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048008+0000","last_change":"2024-02-07T13:09:59.062887+0000","last_active":"2024-02-07T13:10:27.048008+0000","last_peered":"2024-02-07T13:10:27.048008+0000","last_clean":"2024-02-07T13:10:27.048008+0000","last_became_active":"2024-02-07T13:09:59.062488+0000","last_became_peered":"2024-02-07T13:09:59.062488+0000","last_unstale":"2024-02-07T13:10:27.048008+0000","last_undegraded":"2024-02-07T13:10:27.048008+0000","last_fullsized":"2024-02-07T13:10:27.048008+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:23:19.523618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048649+0000","last_change":"2024-02-07T13:10:03.088567+0000","last_active":"2024-02-07T13:10:27.048649+0000","last_peered":"2024-02-07T13:10:27.048649+0000","last_clean":"2024-02-07T13:10:27.048649+0000","last_became_active":"2024-02-07T13:10:03.088241+0000","last_became_peered":"2024-02-07T13:10:03.088241+0000","last_unstale":"2024-02-07T13:10:27.048649+0000","last_undegraded":"2024-02-07T13:10:27.048649+0000","last_fullsized":"2024-02-07T13:10:27.048649+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:02:06.349020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048237+0000","last_change":"2024-02-07T13:10:01.070847+0000","last_active":"2024-02-07T13:10:27.048237+0000","last_peered":"2024-02-07T13:10:27.048237+0000","last_clean":"2024-02-07T13:10:27.048237+0000","last_became_active":"2024-02-07T13:10:01.070110+0000","last_became_peered":"2024-02-07T13:10:01.070110+0000","last_unstale":"2024-02-07T13:10:27.048237+0000","last_undegraded":"2024-02-07T13:10:27.048237+0000","last_fullsized":"2024-02-07T13:10:27.048237+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:59:26.226974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047582+0000","last_change":"2024-02-07T13:09:57.052334+0000","last_active":"2024-02-07T13:10:27.047582+0000","last_peered":"2024-02-07T13:10:27.047582+0000","last_clean":"2024-02-07T13:10:27.047582+0000","last_became_active":"2024-02-07T13:09:57.052004+0000","last_became_peered":"2024-02-07T13:09:57.052004+0000","last_unstale":"2024-02-07T13:10:27.047582+0000","last_undegraded":"2024-02-07T13:10:27.047582+0000","last_fullsized":"2024-02-07T13:10:27.047582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:46:25.083833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"31'5","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048203+0000","last_change":"2024-02-07T13:09:59.058470+0000","last_active":"2024-02-07T13:10:27.048203+0000","last_peered":"2024-02-07T13:10:27.048203+0000","last_clean":"2024-02-07T13:10:27.048203+0000","last_became_active":"2024-02-07T13:09:59.058073+0000","last_became_peered":"2024-02-07T13:09:59.058073+0000","last_unstale":"2024-02-07T13:10:27.048203+0000","last_undegraded":"2024-02-07T13:10:27.048203+0000","last_fullsized":"2024-02-07T13:10:27.048203+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:53:46.824292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048846+0000","last_change":"2024-02-07T13:10:03.089462+0000","last_active":"2024-02-07T13:10:27.048846+0000","last_peered":"2024-02-07T13:10:27.048846+0000","last_clean":"2024-02-07T13:10:27.048846+0000","last_became_active":"2024-02-07T13:10:03.089156+0000","last_became_peered":"2024-02-07T13:10:03.089156+0000","last_unstale":"2024-02-07T13:10:27.048846+0000","last_undegraded":"2024-02-07T13:10:27.048846+0000","last_fullsized":"2024-02-07T13:10:27.048846+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:04:35.809606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'8","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099723+0000","last_change":"2024-02-07T13:10:01.071307+0000","last_active":"2024-02-07T13:10:47.099723+0000","last_peered":"2024-02-07T13:10:47.099723+0000","last_clean":"2024-02-07T13:10:47.099723+0000","last_became_active":"2024-02-07T13:10:01.070650+0000","last_became_peered":"2024-02-07T13:10:01.070650+0000","last_unstale":"2024-02-07T13:10:47.099723+0000","last_undegraded":"2024-02-07T13:10:47.099723+0000","last_fullsized":"2024-02-07T13:10:47.099723+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:42:16.500508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"24'1","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046247+0000","last_change":"2024-02-07T13:09:57.046324+0000","last_active":"2024-02-07T13:10:27.046247+0000","last_peered":"2024-02-07T13:10:27.046247+0000","last_clean":"2024-02-07T13:10:27.046247+0000","last_became_active":"2024-02-07T13:09:57.045892+0000","last_became_peered":"2024-02-07T13:09:57.045892+0000","last_unstale":"2024-02-07T13:10:27.046247+0000","last_undegraded":"2024-02-07T13:10:27.046247+0000","last_fullsized":"2024-02-07T13:10:27.046247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:48:06.526469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.045782+0000","last_change":"2024-02-07T13:09:59.059070+0000","last_active":"2024-02-07T13:10:27.045782+0000","last_peered":"2024-02-07T13:10:27.045782+0000","last_clean":"2024-02-07T13:10:27.045782+0000","last_became_active":"2024-02-07T13:09:59.058480+0000","last_became_peered":"2024-02-07T13:09:59.058480+0000","last_unstale":"2024-02-07T13:10:27.045782+0000","last_undegraded":"2024-02-07T13:10:27.045782+0000","last_fullsized":"2024-02-07T13:10:27.045782+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:56:59.792412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048655+0000","last_change":"2024-02-07T13:10:03.084631+0000","last_active":"2024-02-07T13:10:27.048655+0000","last_peered":"2024-02-07T13:10:27.048655+0000","last_clean":"2024-02-07T13:10:27.048655+0000","last_became_active":"2024-02-07T13:10:03.083922+0000","last_became_peered":"2024-02-07T13:10:03.083922+0000","last_unstale":"2024-02-07T13:10:27.048655+0000","last_undegraded":"2024-02-07T13:10:27.048655+0000","last_fullsized":"2024-02-07T13:10:27.048655+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:16:44.069054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049144+0000","last_change":"2024-02-07T13:10:01.070082+0000","last_active":"2024-02-07T13:10:27.049144+0000","last_peered":"2024-02-07T13:10:27.049144+0000","last_clean":"2024-02-07T13:10:27.049144+0000","last_became_active":"2024-02-07T13:10:01.069636+0000","last_became_peered":"2024-02-07T13:10:01.069636+0000","last_unstale":"2024-02-07T13:10:27.049144+0000","last_undegraded":"2024-02-07T13:10:27.049144+0000","last_fullsized":"2024-02-07T13:10:27.049144+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:33:00.277242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047217+0000","last_change":"2024-02-07T13:09:57.046738+0000","last_active":"2024-02-07T13:10:27.047217+0000","last_peered":"2024-02-07T13:10:27.047217+0000","last_clean":"2024-02-07T13:10:27.047217+0000","last_became_active":"2024-02-07T13:09:57.046394+0000","last_became_peered":"2024-02-07T13:09:57.046394+0000","last_unstale":"2024-02-07T13:10:27.047217+0000","last_undegraded":"2024-02-07T13:10:27.047217+0000","last_fullsized":"2024-02-07T13:10:27.047217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:58:59.957430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"31'15","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.045902+0000","last_change":"2024-02-07T13:09:59.060184+0000","last_active":"2024-02-07T13:10:27.045902+0000","last_peered":"2024-02-07T13:10:27.045902+0000","last_clean":"2024-02-07T13:10:27.045902+0000","last_became_active":"2024-02-07T13:09:59.059858+0000","last_became_peered":"2024-02-07T13:09:59.059858+0000","last_unstale":"2024-02-07T13:10:27.045902+0000","last_undegraded":"2024-02-07T13:10:27.045902+0000","last_fullsized":"2024-02-07T13:10:27.045902+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:44:25.690571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047451+0000","last_change":"2024-02-07T13:10:03.083358+0000","last_active":"2024-02-07T13:10:27.047451+0000","last_peered":"2024-02-07T13:10:27.047451+0000","last_clean":"2024-02-07T13:10:27.047451+0000","last_became_active":"2024-02-07T13:10:03.082915+0000","last_became_peered":"2024-02-07T13:10:03.082915+0000","last_unstale":"2024-02-07T13:10:27.047451+0000","last_undegraded":"2024-02-07T13:10:27.047451+0000","last_fullsized":"2024-02-07T13:10:27.047451+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:53:06.675693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.045799+0000","last_change":"2024-02-07T13:10:01.069233+0000","last_active":"2024-02-07T13:10:27.045799+0000","last_peered":"2024-02-07T13:10:27.045799+0000","last_clean":"2024-02-07T13:10:27.045799+0000","last_became_active":"2024-02-07T13:10:01.068869+0000","last_became_peered":"2024-02-07T13:10:01.068869+0000","last_unstale":"2024-02-07T13:10:27.045799+0000","last_undegraded":"2024-02-07T13:10:27.045799+0000","last_fullsized":"2024-02-07T13:10:27.045799+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:37:24.027498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049664+0000","last_change":"2024-02-07T13:09:57.051664+0000","last_active":"2024-02-07T13:10:27.049664+0000","last_peered":"2024-02-07T13:10:27.049664+0000","last_clean":"2024-02-07T13:10:27.049664+0000","last_became_active":"2024-02-07T13:09:57.051234+0000","last_became_peered":"2024-02-07T13:09:57.051234+0000","last_unstale":"2024-02-07T13:10:27.049664+0000","last_undegraded":"2024-02-07T13:10:27.049664+0000","last_fullsized":"2024-02-07T13:10:27.049664+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:47:30.416264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.050054+0000","last_change":"2024-02-07T13:09:59.059201+0000","last_active":"2024-02-07T13:10:27.050054+0000","last_peered":"2024-02-07T13:10:27.050054+0000","last_clean":"2024-02-07T13:10:27.050054+0000","last_became_active":"2024-02-07T13:09:59.058685+0000","last_became_peered":"2024-02-07T13:09:59.058685+0000","last_unstale":"2024-02-07T13:10:27.050054+0000","last_undegraded":"2024-02-07T13:10:27.050054+0000","last_fullsized":"2024-02-07T13:10:27.050054+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T01:07:06.590953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047225+0000","last_change":"2024-02-07T13:10:03.083280+0000","last_active":"2024-02-07T13:10:27.047225+0000","last_peered":"2024-02-07T13:10:27.047225+0000","last_clean":"2024-02-07T13:10:27.047225+0000","last_became_active":"2024-02-07T13:10:03.082779+0000","last_became_peered":"2024-02-07T13:10:03.082779+0000","last_unstale":"2024-02-07T13:10:27.047225+0000","last_undegraded":"2024-02-07T13:10:27.047225+0000","last_fullsized":"2024-02-07T13:10:27.047225+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:54:23.210248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046172+0000","last_change":"2024-02-07T13:10:01.068136+0000","last_active":"2024-02-07T13:10:27.046172+0000","last_peered":"2024-02-07T13:10:27.046172+0000","last_clean":"2024-02-07T13:10:27.046172+0000","last_became_active":"2024-02-07T13:10:01.067735+0000","last_became_peered":"2024-02-07T13:10:01.067735+0000","last_unstale":"2024-02-07T13:10:27.046172+0000","last_undegraded":"2024-02-07T13:10:27.046172+0000","last_fullsized":"2024-02-07T13:10:27.046172+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:13:08.582408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049841+0000","last_change":"2024-02-07T13:09:57.051248+0000","last_active":"2024-02-07T13:10:27.049841+0000","last_peered":"2024-02-07T13:10:27.049841+0000","last_clean":"2024-02-07T13:10:27.049841+0000","last_became_active":"2024-02-07T13:09:57.050765+0000","last_became_peered":"2024-02-07T13:09:57.050765+0000","last_unstale":"2024-02-07T13:10:27.049841+0000","last_undegraded":"2024-02-07T13:10:27.049841+0000","last_fullsized":"2024-02-07T13:10:27.049841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:16:31.053960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"31'6","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049677+0000","last_change":"2024-02-07T13:09:59.060703+0000","last_active":"2024-02-07T13:10:27.049677+0000","last_peered":"2024-02-07T13:10:27.049677+0000","last_clean":"2024-02-07T13:10:27.049677+0000","last_became_active":"2024-02-07T13:09:59.060407+0000","last_became_peered":"2024-02-07T13:09:59.060407+0000","last_unstale":"2024-02-07T13:10:27.049677+0000","last_undegraded":"2024-02-07T13:10:27.049677+0000","last_fullsized":"2024-02-07T13:10:27.049677+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:35:13.749039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047371+0000","last_change":"2024-02-07T13:10:03.084806+0000","last_active":"2024-02-07T13:10:27.047371+0000","last_peered":"2024-02-07T13:10:27.047371+0000","last_clean":"2024-02-07T13:10:27.047371+0000","last_became_active":"2024-02-07T13:10:03.084461+0000","last_became_peered":"2024-02-07T13:10:03.084461+0000","last_unstale":"2024-02-07T13:10:27.047371+0000","last_undegraded":"2024-02-07T13:10:27.047371+0000","last_fullsized":"2024-02-07T13:10:27.047371+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:07:16.594572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048153+0000","last_change":"2024-02-07T13:10:01.069477+0000","last_active":"2024-02-07T13:10:27.048153+0000","last_peered":"2024-02-07T13:10:27.048153+0000","last_clean":"2024-02-07T13:10:27.048153+0000","last_became_active":"2024-02-07T13:10:01.069137+0000","last_became_peered":"2024-02-07T13:10:01.069137+0000","last_unstale":"2024-02-07T13:10:27.048153+0000","last_undegraded":"2024-02-07T13:10:27.048153+0000","last_fullsized":"2024-02-07T13:10:27.048153+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:59:38.130120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049514+0000","last_change":"2024-02-07T13:09:57.049940+0000","last_active":"2024-02-07T13:10:27.049514+0000","last_peered":"2024-02-07T13:10:27.049514+0000","last_clean":"2024-02-07T13:10:27.049514+0000","last_became_active":"2024-02-07T13:09:57.048046+0000","last_became_peered":"2024-02-07T13:09:57.048046+0000","last_unstale":"2024-02-07T13:10:27.049514+0000","last_undegraded":"2024-02-07T13:10:27.049514+0000","last_fullsized":"2024-02-07T13:10:27.049514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:42:39.845051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"31'10","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049223+0000","last_change":"2024-02-07T13:09:59.057613+0000","last_active":"2024-02-07T13:10:27.049223+0000","last_peered":"2024-02-07T13:10:27.049223+0000","last_clean":"2024-02-07T13:10:27.049223+0000","last_became_active":"2024-02-07T13:09:59.057365+0000","last_became_peered":"2024-02-07T13:09:59.057365+0000","last_unstale":"2024-02-07T13:10:27.049223+0000","last_undegraded":"2024-02-07T13:10:27.049223+0000","last_fullsized":"2024-02-07T13:10:27.049223+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:18:45.692893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047345+0000","last_change":"2024-02-07T13:10:03.084409+0000","last_active":"2024-02-07T13:10:27.047345+0000","last_peered":"2024-02-07T13:10:27.047345+0000","last_clean":"2024-02-07T13:10:27.047345+0000","last_became_active":"2024-02-07T13:10:03.084127+0000","last_became_peered":"2024-02-07T13:10:03.084127+0000","last_unstale":"2024-02-07T13:10:27.047345+0000","last_undegraded":"2024-02-07T13:10:27.047345+0000","last_fullsized":"2024-02-07T13:10:27.047345+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:03:02.620255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046379+0000","last_change":"2024-02-07T13:10:01.067967+0000","last_active":"2024-02-07T13:10:27.046379+0000","last_peered":"2024-02-07T13:10:27.046379+0000","last_clean":"2024-02-07T13:10:27.046379+0000","last_became_active":"2024-02-07T13:10:01.067674+0000","last_became_peered":"2024-02-07T13:10:01.067674+0000","last_unstale":"2024-02-07T13:10:27.046379+0000","last_undegraded":"2024-02-07T13:10:27.046379+0000","last_fullsized":"2024-02-07T13:10:27.046379+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:16:26.922137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048990+0000","last_change":"2024-02-07T13:09:57.051648+0000","last_active":"2024-02-07T13:10:27.048990+0000","last_peered":"2024-02-07T13:10:27.048990+0000","last_clean":"2024-02-07T13:10:27.048990+0000","last_became_active":"2024-02-07T13:09:57.051316+0000","last_became_peered":"2024-02-07T13:09:57.051316+0000","last_unstale":"2024-02-07T13:10:27.048990+0000","last_undegraded":"2024-02-07T13:10:27.048990+0000","last_fullsized":"2024-02-07T13:10:27.048990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:49:46.881103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049324+0000","last_change":"2024-02-07T13:09:59.058450+0000","last_active":"2024-02-07T13:10:27.049324+0000","last_peered":"2024-02-07T13:10:27.049324+0000","last_clean":"2024-02-07T13:10:27.049324+0000","last_became_active":"2024-02-07T13:09:59.058181+0000","last_became_peered":"2024-02-07T13:09:59.058181+0000","last_unstale":"2024-02-07T13:10:27.049324+0000","last_undegraded":"2024-02-07T13:10:27.049324+0000","last_fullsized":"2024-02-07T13:10:27.049324+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:52:07.701473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047565+0000","last_change":"2024-02-07T13:10:03.088471+0000","last_active":"2024-02-07T13:10:27.047565+0000","last_peered":"2024-02-07T13:10:27.047565+0000","last_clean":"2024-02-07T13:10:27.047565+0000","last_became_active":"2024-02-07T13:10:03.088045+0000","last_became_peered":"2024-02-07T13:10:03.088045+0000","last_unstale":"2024-02-07T13:10:27.047565+0000","last_undegraded":"2024-02-07T13:10:27.047565+0000","last_fullsized":"2024-02-07T13:10:27.047565+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:09:33.150941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"31'8","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099678+0000","last_change":"2024-02-07T13:10:01.067694+0000","last_active":"2024-02-07T13:10:47.099678+0000","last_peered":"2024-02-07T13:10:47.099678+0000","last_clean":"2024-02-07T13:10:47.099678+0000","last_became_active":"2024-02-07T13:10:01.067382+0000","last_became_peered":"2024-02-07T13:10:01.067382+0000","last_unstale":"2024-02-07T13:10:47.099678+0000","last_undegraded":"2024-02-07T13:10:47.099678+0000","last_fullsized":"2024-02-07T13:10:47.099678+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:46:40.939112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048017+0000","last_change":"2024-02-07T13:09:57.049270+0000","last_active":"2024-02-07T13:10:27.048017+0000","last_peered":"2024-02-07T13:10:27.048017+0000","last_clean":"2024-02-07T13:10:27.048017+0000","last_became_active":"2024-02-07T13:09:57.048114+0000","last_became_peered":"2024-02-07T13:09:57.048114+0000","last_unstale":"2024-02-07T13:10:27.048017+0000","last_undegraded":"2024-02-07T13:10:27.048017+0000","last_fullsized":"2024-02-07T13:10:27.048017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:12:14.607256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046592+0000","last_change":"2024-02-07T13:09:59.058720+0000","last_active":"2024-02-07T13:10:27.046592+0000","last_peered":"2024-02-07T13:10:27.046592+0000","last_clean":"2024-02-07T13:10:27.046592+0000","last_became_active":"2024-02-07T13:09:59.058371+0000","last_became_peered":"2024-02-07T13:09:59.058371+0000","last_unstale":"2024-02-07T13:10:27.046592+0000","last_undegraded":"2024-02-07T13:10:27.046592+0000","last_fullsized":"2024-02-07T13:10:27.046592+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:34:08.111432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048353+0000","last_change":"2024-02-07T13:10:03.085577+0000","last_active":"2024-02-07T13:10:27.048353+0000","last_peered":"2024-02-07T13:10:27.048353+0000","last_clean":"2024-02-07T13:10:27.048353+0000","last_became_active":"2024-02-07T13:10:03.085269+0000","last_became_peered":"2024-02-07T13:10:03.085269+0000","last_unstale":"2024-02-07T13:10:27.048353+0000","last_undegraded":"2024-02-07T13:10:27.048353+0000","last_fullsized":"2024-02-07T13:10:27.048353+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:39:05.976236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"31'8","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099699+0000","last_change":"2024-02-07T13:10:01.070016+0000","last_active":"2024-02-07T13:10:47.099699+0000","last_peered":"2024-02-07T13:10:47.099699+0000","last_clean":"2024-02-07T13:10:47.099699+0000","last_became_active":"2024-02-07T13:10:01.069713+0000","last_became_peered":"2024-02-07T13:10:01.069713+0000","last_unstale":"2024-02-07T13:10:47.099699+0000","last_undegraded":"2024-02-07T13:10:47.099699+0000","last_fullsized":"2024-02-07T13:10:47.099699+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:34:21.958265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"24'1","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047631+0000","last_change":"2024-02-07T13:09:57.048109+0000","last_active":"2024-02-07T13:10:27.047631+0000","last_peered":"2024-02-07T13:10:27.047631+0000","last_clean":"2024-02-07T13:10:27.047631+0000","last_became_active":"2024-02-07T13:09:57.046824+0000","last_became_peered":"2024-02-07T13:09:57.046824+0000","last_unstale":"2024-02-07T13:10:27.047631+0000","last_undegraded":"2024-02-07T13:10:27.047631+0000","last_fullsized":"2024-02-07T13:10:27.047631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:01:09.822849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"31'11","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048784+0000","last_change":"2024-02-07T13:09:59.056921+0000","last_active":"2024-02-07T13:10:27.048784+0000","last_peered":"2024-02-07T13:10:27.048784+0000","last_clean":"2024-02-07T13:10:27.048784+0000","last_became_active":"2024-02-07T13:09:59.056619+0000","last_became_peered":"2024-02-07T13:09:59.056619+0000","last_unstale":"2024-02-07T13:10:27.048784+0000","last_undegraded":"2024-02-07T13:10:27.048784+0000","last_fullsized":"2024-02-07T13:10:27.048784+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:40:39.394443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049465+0000","last_change":"2024-02-07T13:10:03.084939+0000","last_active":"2024-02-07T13:10:27.049465+0000","last_peered":"2024-02-07T13:10:27.049465+0000","last_clean":"2024-02-07T13:10:27.049465+0000","last_became_active":"2024-02-07T13:10:03.083993+0000","last_became_peered":"2024-02-07T13:10:03.083993+0000","last_unstale":"2024-02-07T13:10:27.049465+0000","last_undegraded":"2024-02-07T13:10:27.049465+0000","last_fullsized":"2024-02-07T13:10:27.049465+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:50:15.026004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049086+0000","last_change":"2024-02-07T13:10:01.071023+0000","last_active":"2024-02-07T13:10:27.049086+0000","last_peered":"2024-02-07T13:10:27.049086+0000","last_clean":"2024-02-07T13:10:27.049086+0000","last_became_active":"2024-02-07T13:10:01.070034+0000","last_became_peered":"2024-02-07T13:10:01.070034+0000","last_unstale":"2024-02-07T13:10:27.049086+0000","last_undegraded":"2024-02-07T13:10:27.049086+0000","last_fullsized":"2024-02-07T13:10:27.049086+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:31:14.193163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047379+0000","last_change":"2024-02-07T13:09:57.051282+0000","last_active":"2024-02-07T13:10:27.047379+0000","last_peered":"2024-02-07T13:10:27.047379+0000","last_clean":"2024-02-07T13:10:27.047379+0000","last_became_active":"2024-02-07T13:09:57.050804+0000","last_became_peered":"2024-02-07T13:09:57.050804+0000","last_unstale":"2024-02-07T13:10:27.047379+0000","last_undegraded":"2024-02-07T13:10:27.047379+0000","last_fullsized":"2024-02-07T13:10:27.047379+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:29:10.880281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"31'4","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046909+0000","last_change":"2024-02-07T13:09:59.062482+0000","last_active":"2024-02-07T13:10:27.046909+0000","last_peered":"2024-02-07T13:10:27.046909+0000","last_clean":"2024-02-07T13:10:27.046909+0000","last_became_active":"2024-02-07T13:09:59.062176+0000","last_became_peered":"2024-02-07T13:09:59.062176+0000","last_unstale":"2024-02-07T13:10:27.046909+0000","last_undegraded":"2024-02-07T13:10:27.046909+0000","last_fullsized":"2024-02-07T13:10:27.046909+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:49:54.976113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048971+0000","last_change":"2024-02-07T13:10:03.083917+0000","last_active":"2024-02-07T13:10:27.048971+0000","last_peered":"2024-02-07T13:10:27.048971+0000","last_clean":"2024-02-07T13:10:27.048971+0000","last_became_active":"2024-02-07T13:10:03.083203+0000","last_became_peered":"2024-02-07T13:10:03.083203+0000","last_unstale":"2024-02-07T13:10:27.048971+0000","last_undegraded":"2024-02-07T13:10:27.048971+0000","last_fullsized":"2024-02-07T13:10:27.048971+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:51:21.941839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049236+0000","last_change":"2024-02-07T13:10:01.071481+0000","last_active":"2024-02-07T13:10:27.049236+0000","last_peered":"2024-02-07T13:10:27.049236+0000","last_clean":"2024-02-07T13:10:27.049236+0000","last_became_active":"2024-02-07T13:10:01.071067+0000","last_became_peered":"2024-02-07T13:10:01.071067+0000","last_unstale":"2024-02-07T13:10:27.049236+0000","last_undegraded":"2024-02-07T13:10:27.049236+0000","last_fullsized":"2024-02-07T13:10:27.049236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:42:42.837022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047321+0000","last_change":"2024-02-07T13:09:57.050043+0000","last_active":"2024-02-07T13:10:27.047321+0000","last_peered":"2024-02-07T13:10:27.047321+0000","last_clean":"2024-02-07T13:10:27.047321+0000","last_became_active":"2024-02-07T13:09:57.049253+0000","last_became_peered":"2024-02-07T13:09:57.049253+0000","last_unstale":"2024-02-07T13:10:27.047321+0000","last_undegraded":"2024-02-07T13:10:27.047321+0000","last_fullsized":"2024-02-07T13:10:27.047321+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:19:44.671027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"31'11","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048230+0000","last_change":"2024-02-07T13:09:59.058588+0000","last_active":"2024-02-07T13:10:27.048230+0000","last_peered":"2024-02-07T13:10:27.048230+0000","last_clean":"2024-02-07T13:10:27.048230+0000","last_became_active":"2024-02-07T13:09:59.058179+0000","last_became_peered":"2024-02-07T13:09:59.058179+0000","last_unstale":"2024-02-07T13:10:27.048230+0000","last_undegraded":"2024-02-07T13:10:27.048230+0000","last_fullsized":"2024-02-07T13:10:27.048230+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:13:25.374623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048344+0000","last_change":"2024-02-07T13:10:03.087474+0000","last_active":"2024-02-07T13:10:27.048344+0000","last_peered":"2024-02-07T13:10:27.048344+0000","last_clean":"2024-02-07T13:10:27.048344+0000","last_became_active":"2024-02-07T13:10:03.086938+0000","last_became_peered":"2024-02-07T13:10:03.086938+0000","last_unstale":"2024-02-07T13:10:27.048344+0000","last_undegraded":"2024-02-07T13:10:27.048344+0000","last_fullsized":"2024-02-07T13:10:27.048344+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:21:39.226846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"24'2","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049480+0000","last_change":"2024-02-07T13:09:57.048265+0000","last_active":"2024-02-07T13:10:27.049480+0000","last_peered":"2024-02-07T13:10:27.049480+0000","last_clean":"2024-02-07T13:10:27.049480+0000","last_became_active":"2024-02-07T13:09:57.046991+0000","last_became_peered":"2024-02-07T13:09:57.046991+0000","last_unstale":"2024-02-07T13:10:27.049480+0000","last_undegraded":"2024-02-07T13:10:27.049480+0000","last_fullsized":"2024-02-07T13:10:27.049480+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:07:59.205642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"31'11","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049628+0000","last_change":"2024-02-07T13:09:59.062964+0000","last_active":"2024-02-07T13:10:27.049628+0000","last_peered":"2024-02-07T13:10:27.049628+0000","last_clean":"2024-02-07T13:10:27.049628+0000","last_became_active":"2024-02-07T13:09:59.062609+0000","last_became_peered":"2024-02-07T13:09:59.062609+0000","last_unstale":"2024-02-07T13:10:27.049628+0000","last_undegraded":"2024-02-07T13:10:27.049628+0000","last_fullsized":"2024-02-07T13:10:27.049628+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:01:56.347117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'8","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099749+0000","last_change":"2024-02-07T13:10:01.067654+0000","last_active":"2024-02-07T13:10:47.099749+0000","last_peered":"2024-02-07T13:10:47.099749+0000","last_clean":"2024-02-07T13:10:47.099749+0000","last_became_active":"2024-02-07T13:10:01.067289+0000","last_became_peered":"2024-02-07T13:10:01.067289+0000","last_unstale":"2024-02-07T13:10:47.099749+0000","last_undegraded":"2024-02-07T13:10:47.099749+0000","last_fullsized":"2024-02-07T13:10:47.099749+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:37:37.914427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046551+0000","last_change":"2024-02-07T13:10:03.084064+0000","last_active":"2024-02-07T13:10:27.046551+0000","last_peered":"2024-02-07T13:10:27.046551+0000","last_clean":"2024-02-07T13:10:27.046551+0000","last_became_active":"2024-02-07T13:10:03.083677+0000","last_became_peered":"2024-02-07T13:10:03.083677+0000","last_unstale":"2024-02-07T13:10:27.046551+0000","last_undegraded":"2024-02-07T13:10:27.046551+0000","last_fullsized":"2024-02-07T13:10:27.046551+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:04:48.581898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048216+0000","last_change":"2024-02-07T13:09:57.046631+0000","last_active":"2024-02-07T13:10:27.048216+0000","last_peered":"2024-02-07T13:10:27.048216+0000","last_clean":"2024-02-07T13:10:27.048216+0000","last_became_active":"2024-02-07T13:09:57.046183+0000","last_became_peered":"2024-02-07T13:09:57.046183+0000","last_unstale":"2024-02-07T13:10:27.048216+0000","last_undegraded":"2024-02-07T13:10:27.048216+0000","last_fullsized":"2024-02-07T13:10:27.048216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:08:59.572003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"31'16","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099577+0000","last_change":"2024-02-07T13:09:59.058819+0000","last_active":"2024-02-07T13:10:47.099577+0000","last_peered":"2024-02-07T13:10:47.099577+0000","last_clean":"2024-02-07T13:10:47.099577+0000","last_became_active":"2024-02-07T13:09:59.058424+0000","last_became_peered":"2024-02-07T13:09:59.058424+0000","last_unstale":"2024-02-07T13:10:47.099577+0000","last_undegraded":"2024-02-07T13:10:47.099577+0000","last_fullsized":"2024-02-07T13:10:47.099577+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:47:51.736035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047466+0000","last_change":"2024-02-07T13:10:01.069928+0000","last_active":"2024-02-07T13:10:27.047466+0000","last_peered":"2024-02-07T13:10:27.047466+0000","last_clean":"2024-02-07T13:10:27.047466+0000","last_became_active":"2024-02-07T13:10:01.069645+0000","last_became_peered":"2024-02-07T13:10:01.069645+0000","last_unstale":"2024-02-07T13:10:27.047466+0000","last_undegraded":"2024-02-07T13:10:27.047466+0000","last_fullsized":"2024-02-07T13:10:27.047466+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:15:12.369561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048151+0000","last_change":"2024-02-07T13:10:03.084303+0000","last_active":"2024-02-07T13:10:27.048151+0000","last_peered":"2024-02-07T13:10:27.048151+0000","last_clean":"2024-02-07T13:10:27.048151+0000","last_became_active":"2024-02-07T13:10:03.083952+0000","last_became_peered":"2024-02-07T13:10:03.083952+0000","last_unstale":"2024-02-07T13:10:27.048151+0000","last_undegraded":"2024-02-07T13:10:27.048151+0000","last_fullsized":"2024-02-07T13:10:27.048151+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:36:58.824048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"24'1","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047737+0000","last_change":"2024-02-07T13:09:57.051899+0000","last_active":"2024-02-07T13:10:27.047737+0000","last_peered":"2024-02-07T13:10:27.047737+0000","last_clean":"2024-02-07T13:10:27.047737+0000","last_became_active":"2024-02-07T13:09:57.051533+0000","last_became_peered":"2024-02-07T13:09:57.051533+0000","last_unstale":"2024-02-07T13:10:27.047737+0000","last_undegraded":"2024-02-07T13:10:27.047737+0000","last_fullsized":"2024-02-07T13:10:27.047737+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:47:57.739666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"31'19","reported_seq":59,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048508+0000","last_change":"2024-02-07T13:09:59.056897+0000","last_active":"2024-02-07T13:10:27.048508+0000","last_peered":"2024-02-07T13:10:27.048508+0000","last_clean":"2024-02-07T13:10:27.048508+0000","last_became_active":"2024-02-07T13:09:59.056509+0000","last_became_peered":"2024-02-07T13:09:59.056509+0000","last_unstale":"2024-02-07T13:10:27.048508+0000","last_undegraded":"2024-02-07T13:10:27.048508+0000","last_fullsized":"2024-02-07T13:10:27.048508+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:35:46.936987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047648+0000","last_change":"2024-02-07T13:10:01.069820+0000","last_active":"2024-02-07T13:10:27.047648+0000","last_peered":"2024-02-07T13:10:27.047648+0000","last_clean":"2024-02-07T13:10:27.047648+0000","last_became_active":"2024-02-07T13:10:01.069514+0000","last_became_peered":"2024-02-07T13:10:01.069514+0000","last_unstale":"2024-02-07T13:10:27.047648+0000","last_undegraded":"2024-02-07T13:10:27.047648+0000","last_fullsized":"2024-02-07T13:10:27.047648+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:56:46.538421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049679+0000","last_change":"2024-02-07T13:10:03.089611+0000","last_active":"2024-02-07T13:10:27.049679+0000","last_peered":"2024-02-07T13:10:27.049679+0000","last_clean":"2024-02-07T13:10:27.049679+0000","last_became_active":"2024-02-07T13:10:03.089206+0000","last_became_peered":"2024-02-07T13:10:03.089206+0000","last_unstale":"2024-02-07T13:10:27.049679+0000","last_undegraded":"2024-02-07T13:10:27.049679+0000","last_fullsized":"2024-02-07T13:10:27.049679+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:56:50.474614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"31'18","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047018+0000","last_change":"2024-02-07T13:09:59.063025+0000","last_active":"2024-02-07T13:10:27.047018+0000","last_peered":"2024-02-07T13:10:27.047018+0000","last_clean":"2024-02-07T13:10:27.047018+0000","last_became_active":"2024-02-07T13:09:59.062476+0000","last_became_peered":"2024-02-07T13:09:59.062476+0000","last_unstale":"2024-02-07T13:10:27.047018+0000","last_undegraded":"2024-02-07T13:10:27.047018+0000","last_fullsized":"2024-02-07T13:10:27.047018+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:49:30.957915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046017+0000","last_change":"2024-02-07T13:09:57.044747+0000","last_active":"2024-02-07T13:10:27.046017+0000","last_peered":"2024-02-07T13:10:27.046017+0000","last_clean":"2024-02-07T13:10:27.046017+0000","last_became_active":"2024-02-07T13:09:57.044366+0000","last_became_peered":"2024-02-07T13:09:57.044366+0000","last_unstale":"2024-02-07T13:10:27.046017+0000","last_undegraded":"2024-02-07T13:10:27.046017+0000","last_fullsized":"2024-02-07T13:10:27.046017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:45:04.878046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049216+0000","last_change":"2024-02-07T13:10:01.074657+0000","last_active":"2024-02-07T13:10:27.049216+0000","last_peered":"2024-02-07T13:10:27.049216+0000","last_clean":"2024-02-07T13:10:27.049216+0000","last_became_active":"2024-02-07T13:10:01.074241+0000","last_became_peered":"2024-02-07T13:10:01.074241+0000","last_unstale":"2024-02-07T13:10:27.049216+0000","last_undegraded":"2024-02-07T13:10:27.049216+0000","last_fullsized":"2024-02-07T13:10:27.049216+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:55:29.042308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047581+0000","last_change":"2024-02-07T13:10:03.087241+0000","last_active":"2024-02-07T13:10:27.047581+0000","last_peered":"2024-02-07T13:10:27.047581+0000","last_clean":"2024-02-07T13:10:27.047581+0000","last_became_active":"2024-02-07T13:10:03.086928+0000","last_became_peered":"2024-02-07T13:10:03.086928+0000","last_unstale":"2024-02-07T13:10:27.047581+0000","last_undegraded":"2024-02-07T13:10:27.047581+0000","last_fullsized":"2024-02-07T13:10:27.047581+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:39:10.461534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"31'10","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046466+0000","last_change":"2024-02-07T13:09:59.058663+0000","last_active":"2024-02-07T13:10:27.046466+0000","last_peered":"2024-02-07T13:10:27.046466+0000","last_clean":"2024-02-07T13:10:27.046466+0000","last_became_active":"2024-02-07T13:09:59.058350+0000","last_became_peered":"2024-02-07T13:09:59.058350+0000","last_unstale":"2024-02-07T13:10:27.046466+0000","last_undegraded":"2024-02-07T13:10:27.046466+0000","last_fullsized":"2024-02-07T13:10:27.046466+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T01:05:26.891836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048268+0000","last_change":"2024-02-07T13:09:57.052708+0000","last_active":"2024-02-07T13:10:27.048268+0000","last_peered":"2024-02-07T13:10:27.048268+0000","last_clean":"2024-02-07T13:10:27.048268+0000","last_became_active":"2024-02-07T13:09:57.052338+0000","last_became_peered":"2024-02-07T13:09:57.052338+0000","last_unstale":"2024-02-07T13:10:27.048268+0000","last_undegraded":"2024-02-07T13:10:27.048268+0000","last_fullsized":"2024-02-07T13:10:27.048268+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:54:49.812290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"32'208","reported_seq":314,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:29.160902+0000","last_change":"2024-02-07T13:09:47.188747+0000","last_active":"2024-02-07T13:10:29.160902+0000","last_peered":"2024-02-07T13:10:29.160902+0000","last_clean":"2024-02-07T13:10:29.160902+0000","last_became_active":"2024-02-07T13:09:47.188364+0000","last_became_peered":"2024-02-07T13:09:47.188364+0000","last_unstale":"2024-02-07T13:10:29.160902+0000","last_undegraded":"2024-02-07T13:10:29.160902+0000","last_fullsized":"2024-02-07T13:10:29.160902+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:46.169165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:46.169165+0000","last_clean_scrub_stamp":"2024-02-07T13:09:46.169165+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:42:48.929160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048825+0000","last_change":"2024-02-07T13:10:01.069708+0000","last_active":"2024-02-07T13:10:27.048825+0000","last_peered":"2024-02-07T13:10:27.048825+0000","last_clean":"2024-02-07T13:10:27.048825+0000","last_became_active":"2024-02-07T13:10:01.069389+0000","last_became_peered":"2024-02-07T13:10:01.069389+0000","last_unstale":"2024-02-07T13:10:27.048825+0000","last_undegraded":"2024-02-07T13:10:27.048825+0000","last_fullsized":"2024-02-07T13:10:27.048825+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:00:15.026029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048897+0000","last_change":"2024-02-07T13:10:03.086629+0000","last_active":"2024-02-07T13:10:27.048897+0000","last_peered":"2024-02-07T13:10:27.048897+0000","last_clean":"2024-02-07T13:10:27.048897+0000","last_became_active":"2024-02-07T13:10:03.085398+0000","last_became_peered":"2024-02-07T13:10:03.085398+0000","last_unstale":"2024-02-07T13:10:27.048897+0000","last_undegraded":"2024-02-07T13:10:27.048897+0000","last_fullsized":"2024-02-07T13:10:27.048897+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:52:40.751196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"31'14","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046250+0000","last_change":"2024-02-07T13:09:59.059584+0000","last_active":"2024-02-07T13:10:27.046250+0000","last_peered":"2024-02-07T13:10:27.046250+0000","last_clean":"2024-02-07T13:10:27.046250+0000","last_became_active":"2024-02-07T13:09:59.059247+0000","last_became_peered":"2024-02-07T13:09:59.059247+0000","last_unstale":"2024-02-07T13:10:27.046250+0000","last_undegraded":"2024-02-07T13:10:27.046250+0000","last_fullsized":"2024-02-07T13:10:27.046250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:39:48.582929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046997+0000","last_change":"2024-02-07T13:09:57.053260+0000","last_active":"2024-02-07T13:10:27.046997+0000","last_peered":"2024-02-07T13:10:27.046997+0000","last_clean":"2024-02-07T13:10:27.046997+0000","last_became_active":"2024-02-07T13:09:57.052900+0000","last_became_peered":"2024-02-07T13:09:57.052900+0000","last_unstale":"2024-02-07T13:10:27.046997+0000","last_undegraded":"2024-02-07T13:10:27.046997+0000","last_fullsized":"2024-02-07T13:10:27.046997+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:16:28.463232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048202+0000","last_change":"2024-02-07T13:10:01.067894+0000","last_active":"2024-02-07T13:10:27.048202+0000","last_peered":"2024-02-07T13:10:27.048202+0000","last_clean":"2024-02-07T13:10:27.048202+0000","last_became_active":"2024-02-07T13:10:01.067241+0000","last_became_peered":"2024-02-07T13:10:01.067241+0000","last_unstale":"2024-02-07T13:10:27.048202+0000","last_undegraded":"2024-02-07T13:10:27.048202+0000","last_fullsized":"2024-02-07T13:10:27.048202+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:15:30.741385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047641+0000","last_change":"2024-02-07T13:10:03.083440+0000","last_active":"2024-02-07T13:10:27.047641+0000","last_peered":"2024-02-07T13:10:27.047641+0000","last_clean":"2024-02-07T13:10:27.047641+0000","last_became_active":"2024-02-07T13:10:03.082905+0000","last_became_peered":"2024-02-07T13:10:03.082905+0000","last_unstale":"2024-02-07T13:10:27.047641+0000","last_undegraded":"2024-02-07T13:10:27.047641+0000","last_fullsized":"2024-02-07T13:10:27.047641+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:12:50.039813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049448+0000","last_change":"2024-02-07T13:09:57.050153+0000","last_active":"2024-02-07T13:10:27.049448+0000","last_peered":"2024-02-07T13:10:27.049448+0000","last_clean":"2024-02-07T13:10:27.049448+0000","last_became_active":"2024-02-07T13:09:57.049747+0000","last_became_peered":"2024-02-07T13:09:57.049747+0000","last_unstale":"2024-02-07T13:10:27.049448+0000","last_undegraded":"2024-02-07T13:10:27.049448+0000","last_fullsized":"2024-02-07T13:10:27.049448+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:45:59.518112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"31'28","reported_seq":83,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099540+0000","last_change":"2024-02-07T13:09:59.061206+0000","last_active":"2024-02-07T13:10:47.099540+0000","last_peered":"2024-02-07T13:10:47.099540+0000","last_clean":"2024-02-07T13:10:47.099540+0000","last_became_active":"2024-02-07T13:09:59.060778+0000","last_became_peered":"2024-02-07T13:09:59.060778+0000","last_unstale":"2024-02-07T13:10:47.099540+0000","last_undegraded":"2024-02-07T13:10:47.099540+0000","last_fullsized":"2024-02-07T13:10:47.099540+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:10:00.044490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"31'8","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099735+0000","last_change":"2024-02-07T13:10:01.069065+0000","last_active":"2024-02-07T13:10:47.099735+0000","last_peered":"2024-02-07T13:10:47.099735+0000","last_clean":"2024-02-07T13:10:47.099735+0000","last_became_active":"2024-02-07T13:10:01.068194+0000","last_became_peered":"2024-02-07T13:10:01.068194+0000","last_unstale":"2024-02-07T13:10:47.099735+0000","last_undegraded":"2024-02-07T13:10:47.099735+0000","last_fullsized":"2024-02-07T13:10:47.099735+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:35:57.819339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047054+0000","last_change":"2024-02-07T13:10:03.084644+0000","last_active":"2024-02-07T13:10:27.047054+0000","last_peered":"2024-02-07T13:10:27.047054+0000","last_clean":"2024-02-07T13:10:27.047054+0000","last_became_active":"2024-02-07T13:10:03.084040+0000","last_became_peered":"2024-02-07T13:10:03.084040+0000","last_unstale":"2024-02-07T13:10:27.047054+0000","last_undegraded":"2024-02-07T13:10:27.047054+0000","last_fullsized":"2024-02-07T13:10:27.047054+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:36:04.508794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049813+0000","last_change":"2024-02-07T13:09:57.046346+0000","last_active":"2024-02-07T13:10:27.049813+0000","last_peered":"2024-02-07T13:10:27.049813+0000","last_clean":"2024-02-07T13:10:27.049813+0000","last_became_active":"2024-02-07T13:09:57.045714+0000","last_became_peered":"2024-02-07T13:09:57.045714+0000","last_unstale":"2024-02-07T13:10:27.049813+0000","last_undegraded":"2024-02-07T13:10:27.049813+0000","last_fullsized":"2024-02-07T13:10:27.049813+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:30:58.986680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"31'13","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049679+0000","last_change":"2024-02-07T13:09:59.062189+0000","last_active":"2024-02-07T13:10:27.049679+0000","last_peered":"2024-02-07T13:10:27.049679+0000","last_clean":"2024-02-07T13:10:27.049679+0000","last_became_active":"2024-02-07T13:09:59.061919+0000","last_became_peered":"2024-02-07T13:09:59.061919+0000","last_unstale":"2024-02-07T13:10:27.049679+0000","last_undegraded":"2024-02-07T13:10:27.049679+0000","last_fullsized":"2024-02-07T13:10:27.049679+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:35:27.096214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"31'8","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099514+0000","last_change":"2024-02-07T13:10:01.074761+0000","last_active":"2024-02-07T13:10:47.099514+0000","last_peered":"2024-02-07T13:10:47.099514+0000","last_clean":"2024-02-07T13:10:47.099514+0000","last_became_active":"2024-02-07T13:10:01.074293+0000","last_became_peered":"2024-02-07T13:10:01.074293+0000","last_unstale":"2024-02-07T13:10:47.099514+0000","last_undegraded":"2024-02-07T13:10:47.099514+0000","last_fullsized":"2024-02-07T13:10:47.099514+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:02:14.719921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047868+0000","last_change":"2024-02-07T13:10:03.085019+0000","last_active":"2024-02-07T13:10:27.047868+0000","last_peered":"2024-02-07T13:10:27.047868+0000","last_clean":"2024-02-07T13:10:27.047868+0000","last_became_active":"2024-02-07T13:10:03.084531+0000","last_became_peered":"2024-02-07T13:10:03.084531+0000","last_unstale":"2024-02-07T13:10:27.047868+0000","last_undegraded":"2024-02-07T13:10:27.047868+0000","last_fullsized":"2024-02-07T13:10:27.047868+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:40:51.067114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049284+0000","last_change":"2024-02-07T13:09:57.052011+0000","last_active":"2024-02-07T13:10:27.049284+0000","last_peered":"2024-02-07T13:10:27.049284+0000","last_clean":"2024-02-07T13:10:27.049284+0000","last_became_active":"2024-02-07T13:09:57.051567+0000","last_became_peered":"2024-02-07T13:09:57.051567+0000","last_unstale":"2024-02-07T13:10:27.049284+0000","last_undegraded":"2024-02-07T13:10:27.049284+0000","last_fullsized":"2024-02-07T13:10:27.049284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:55:55.217672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"31'12","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049345+0000","last_change":"2024-02-07T13:09:59.061119+0000","last_active":"2024-02-07T13:10:27.049345+0000","last_peered":"2024-02-07T13:10:27.049345+0000","last_clean":"2024-02-07T13:10:27.049345+0000","last_became_active":"2024-02-07T13:09:59.060310+0000","last_became_peered":"2024-02-07T13:09:59.060310+0000","last_unstale":"2024-02-07T13:10:27.049345+0000","last_undegraded":"2024-02-07T13:10:27.049345+0000","last_fullsized":"2024-02-07T13:10:27.049345+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:57:03.247934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047370+0000","last_change":"2024-02-07T13:10:01.069945+0000","last_active":"2024-02-07T13:10:27.047370+0000","last_peered":"2024-02-07T13:10:27.047370+0000","last_clean":"2024-02-07T13:10:27.047370+0000","last_became_active":"2024-02-07T13:10:01.069571+0000","last_became_peered":"2024-02-07T13:10:01.069571+0000","last_unstale":"2024-02-07T13:10:27.047370+0000","last_undegraded":"2024-02-07T13:10:27.047370+0000","last_fullsized":"2024-02-07T13:10:27.047370+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:24:02.546134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048741+0000","last_change":"2024-02-07T13:10:03.084016+0000","last_active":"2024-02-07T13:10:27.048741+0000","last_peered":"2024-02-07T13:10:27.048741+0000","last_clean":"2024-02-07T13:10:27.048741+0000","last_became_active":"2024-02-07T13:10:03.083682+0000","last_became_peered":"2024-02-07T13:10:03.083682+0000","last_unstale":"2024-02-07T13:10:27.048741+0000","last_undegraded":"2024-02-07T13:10:27.048741+0000","last_fullsized":"2024-02-07T13:10:27.048741+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:27:54.916855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"31'8","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099532+0000","last_change":"2024-02-07T13:10:01.070244+0000","last_active":"2024-02-07T13:10:47.099532+0000","last_peered":"2024-02-07T13:10:47.099532+0000","last_clean":"2024-02-07T13:10:47.099532+0000","last_became_active":"2024-02-07T13:10:01.069936+0000","last_became_peered":"2024-02-07T13:10:01.069936+0000","last_unstale":"2024-02-07T13:10:47.099532+0000","last_undegraded":"2024-02-07T13:10:47.099532+0000","last_fullsized":"2024-02-07T13:10:47.099532+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:01:47.717905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047095+0000","last_change":"2024-02-07T13:09:57.053104+0000","last_active":"2024-02-07T13:10:27.047095+0000","last_peered":"2024-02-07T13:10:27.047095+0000","last_clean":"2024-02-07T13:10:27.047095+0000","last_became_active":"2024-02-07T13:09:57.052636+0000","last_became_peered":"2024-02-07T13:09:57.052636+0000","last_unstale":"2024-02-07T13:10:27.047095+0000","last_undegraded":"2024-02-07T13:10:27.047095+0000","last_fullsized":"2024-02-07T13:10:27.047095+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:39:32.298984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"31'12","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048563+0000","last_change":"2024-02-07T13:09:59.058803+0000","last_active":"2024-02-07T13:10:27.048563+0000","last_peered":"2024-02-07T13:10:27.048563+0000","last_clean":"2024-02-07T13:10:27.048563+0000","last_became_active":"2024-02-07T13:09:59.058507+0000","last_became_peered":"2024-02-07T13:09:59.058507+0000","last_unstale":"2024-02-07T13:10:27.048563+0000","last_undegraded":"2024-02-07T13:10:27.048563+0000","last_fullsized":"2024-02-07T13:10:27.048563+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:32:51.490644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049282+0000","last_change":"2024-02-07T13:10:03.087204+0000","last_active":"2024-02-07T13:10:27.049282+0000","last_peered":"2024-02-07T13:10:27.049282+0000","last_clean":"2024-02-07T13:10:27.049282+0000","last_became_active":"2024-02-07T13:10:03.086868+0000","last_became_peered":"2024-02-07T13:10:03.086868+0000","last_unstale":"2024-02-07T13:10:27.049282+0000","last_undegraded":"2024-02-07T13:10:27.049282+0000","last_fullsized":"2024-02-07T13:10:27.049282+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:23:31.655829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049127+0000","last_change":"2024-02-07T13:10:01.073468+0000","last_active":"2024-02-07T13:10:27.049127+0000","last_peered":"2024-02-07T13:10:27.049127+0000","last_clean":"2024-02-07T13:10:27.049127+0000","last_became_active":"2024-02-07T13:10:01.073188+0000","last_became_peered":"2024-02-07T13:10:01.073188+0000","last_unstale":"2024-02-07T13:10:27.049127+0000","last_undegraded":"2024-02-07T13:10:27.049127+0000","last_fullsized":"2024-02-07T13:10:27.049127+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:29:44.593605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046127+0000","last_change":"2024-02-07T13:09:57.044721+0000","last_active":"2024-02-07T13:10:27.046127+0000","last_peered":"2024-02-07T13:10:27.046127+0000","last_clean":"2024-02-07T13:10:27.046127+0000","last_became_active":"2024-02-07T13:09:57.044373+0000","last_became_peered":"2024-02-07T13:09:57.044373+0000","last_unstale":"2024-02-07T13:10:27.046127+0000","last_undegraded":"2024-02-07T13:10:27.046127+0000","last_fullsized":"2024-02-07T13:10:27.046127+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:30:25.307104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"31'15","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047327+0000","last_change":"2024-02-07T13:09:59.057454+0000","last_active":"2024-02-07T13:10:27.047327+0000","last_peered":"2024-02-07T13:10:27.047327+0000","last_clean":"2024-02-07T13:10:27.047327+0000","last_became_active":"2024-02-07T13:09:59.057125+0000","last_became_peered":"2024-02-07T13:09:59.057125+0000","last_unstale":"2024-02-07T13:10:27.047327+0000","last_undegraded":"2024-02-07T13:10:27.047327+0000","last_fullsized":"2024-02-07T13:10:27.047327+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:22:17.016717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048520+0000","last_change":"2024-02-07T13:10:03.087638+0000","last_active":"2024-02-07T13:10:27.048520+0000","last_peered":"2024-02-07T13:10:27.048520+0000","last_clean":"2024-02-07T13:10:27.048520+0000","last_became_active":"2024-02-07T13:10:03.086919+0000","last_became_peered":"2024-02-07T13:10:03.086919+0000","last_unstale":"2024-02-07T13:10:27.048520+0000","last_undegraded":"2024-02-07T13:10:27.048520+0000","last_fullsized":"2024-02-07T13:10:27.048520+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:02:54.902460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047977+0000","last_change":"2024-02-07T13:09:57.047170+0000","last_active":"2024-02-07T13:10:27.047977+0000","last_peered":"2024-02-07T13:10:27.047977+0000","last_clean":"2024-02-07T13:10:27.047977+0000","last_became_active":"2024-02-07T13:09:57.046766+0000","last_became_peered":"2024-02-07T13:09:57.046766+0000","last_unstale":"2024-02-07T13:10:27.047977+0000","last_undegraded":"2024-02-07T13:10:27.047977+0000","last_fullsized":"2024-02-07T13:10:27.047977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:36:36.062045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046904+0000","last_change":"2024-02-07T13:09:59.059949+0000","last_active":"2024-02-07T13:10:27.046904+0000","last_peered":"2024-02-07T13:10:27.046904+0000","last_clean":"2024-02-07T13:10:27.046904+0000","last_became_active":"2024-02-07T13:09:59.059603+0000","last_became_peered":"2024-02-07T13:09:59.059603+0000","last_unstale":"2024-02-07T13:10:27.046904+0000","last_undegraded":"2024-02-07T13:10:27.046904+0000","last_fullsized":"2024-02-07T13:10:27.046904+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:32:26.800915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048746+0000","last_change":"2024-02-07T13:10:01.069673+0000","last_active":"2024-02-07T13:10:27.048746+0000","last_peered":"2024-02-07T13:10:27.048746+0000","last_clean":"2024-02-07T13:10:27.048746+0000","last_became_active":"2024-02-07T13:10:01.069289+0000","last_became_peered":"2024-02-07T13:10:01.069289+0000","last_unstale":"2024-02-07T13:10:27.048746+0000","last_undegraded":"2024-02-07T13:10:27.048746+0000","last_fullsized":"2024-02-07T13:10:27.048746+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:14:44.669314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047292+0000","last_change":"2024-02-07T13:10:03.084007+0000","last_active":"2024-02-07T13:10:27.047292+0000","last_peered":"2024-02-07T13:10:27.047292+0000","last_clean":"2024-02-07T13:10:27.047292+0000","last_became_active":"2024-02-07T13:10:03.083199+0000","last_became_peered":"2024-02-07T13:10:03.083199+0000","last_unstale":"2024-02-07T13:10:27.047292+0000","last_undegraded":"2024-02-07T13:10:27.047292+0000","last_fullsized":"2024-02-07T13:10:27.047292+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T01:04:59.002423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"31'8","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099489+0000","last_change":"2024-02-07T13:10:01.071049+0000","last_active":"2024-02-07T13:10:47.099489+0000","last_peered":"2024-02-07T13:10:47.099489+0000","last_clean":"2024-02-07T13:10:47.099489+0000","last_became_active":"2024-02-07T13:10:01.070635+0000","last_became_peered":"2024-02-07T13:10:01.070635+0000","last_unstale":"2024-02-07T13:10:47.099489+0000","last_undegraded":"2024-02-07T13:10:47.099489+0000","last_fullsized":"2024-02-07T13:10:47.099489+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:09:34.130416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048536+0000","last_change":"2024-02-07T13:09:57.048948+0000","last_active":"2024-02-07T13:10:27.048536+0000","last_peered":"2024-02-07T13:10:27.048536+0000","last_clean":"2024-02-07T13:10:27.048536+0000","last_became_active":"2024-02-07T13:09:57.047957+0000","last_became_peered":"2024-02-07T13:09:57.047957+0000","last_unstale":"2024-02-07T13:10:27.048536+0000","last_undegraded":"2024-02-07T13:10:27.048536+0000","last_fullsized":"2024-02-07T13:10:27.048536+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:32:57.617385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"31'19","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047911+0000","last_change":"2024-02-07T13:09:59.057023+0000","last_active":"2024-02-07T13:10:27.047911+0000","last_peered":"2024-02-07T13:10:27.047911+0000","last_clean":"2024-02-07T13:10:27.047911+0000","last_became_active":"2024-02-07T13:09:59.056628+0000","last_became_peered":"2024-02-07T13:09:59.056628+0000","last_unstale":"2024-02-07T13:10:27.047911+0000","last_undegraded":"2024-02-07T13:10:27.047911+0000","last_fullsized":"2024-02-07T13:10:27.047911+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:57:05.923715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047617+0000","last_change":"2024-02-07T13:10:03.084832+0000","last_active":"2024-02-07T13:10:27.047617+0000","last_peered":"2024-02-07T13:10:27.047617+0000","last_clean":"2024-02-07T13:10:27.047617+0000","last_became_active":"2024-02-07T13:10:03.084327+0000","last_became_peered":"2024-02-07T13:10:03.084327+0000","last_unstale":"2024-02-07T13:10:27.047617+0000","last_undegraded":"2024-02-07T13:10:27.047617+0000","last_fullsized":"2024-02-07T13:10:27.047617+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:40:14.035731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048642+0000","last_change":"2024-02-07T13:09:57.048052+0000","last_active":"2024-02-07T13:10:27.048642+0000","last_peered":"2024-02-07T13:10:27.048642+0000","last_clean":"2024-02-07T13:10:27.048642+0000","last_became_active":"2024-02-07T13:09:57.046940+0000","last_became_peered":"2024-02-07T13:09:57.046940+0000","last_unstale":"2024-02-07T13:10:27.048642+0000","last_undegraded":"2024-02-07T13:10:27.048642+0000","last_fullsized":"2024-02-07T13:10:27.048642+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:44:53.465201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"31'17","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049251+0000","last_change":"2024-02-07T13:09:59.062479+0000","last_active":"2024-02-07T13:10:27.049251+0000","last_peered":"2024-02-07T13:10:27.049251+0000","last_clean":"2024-02-07T13:10:27.049251+0000","last_became_active":"2024-02-07T13:09:59.062131+0000","last_became_peered":"2024-02-07T13:09:59.062131+0000","last_unstale":"2024-02-07T13:10:27.049251+0000","last_undegraded":"2024-02-07T13:10:27.049251+0000","last_fullsized":"2024-02-07T13:10:27.049251+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:04:07.666771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047482+0000","last_change":"2024-02-07T13:10:01.071535+0000","last_active":"2024-02-07T13:10:27.047482+0000","last_peered":"2024-02-07T13:10:27.047482+0000","last_clean":"2024-02-07T13:10:27.047482+0000","last_became_active":"2024-02-07T13:10:01.071205+0000","last_became_peered":"2024-02-07T13:10:01.071205+0000","last_unstale":"2024-02-07T13:10:27.047482+0000","last_undegraded":"2024-02-07T13:10:27.047482+0000","last_fullsized":"2024-02-07T13:10:27.047482+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:06:34.933639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.045967+0000","last_change":"2024-02-07T13:10:03.082900+0000","last_active":"2024-02-07T13:10:27.045967+0000","last_peered":"2024-02-07T13:10:27.045967+0000","last_clean":"2024-02-07T13:10:27.045967+0000","last_became_active":"2024-02-07T13:10:03.082386+0000","last_became_peered":"2024-02-07T13:10:03.082386+0000","last_unstale":"2024-02-07T13:10:27.045967+0000","last_undegraded":"2024-02-07T13:10:27.045967+0000","last_fullsized":"2024-02-07T13:10:27.045967+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:19:53.241008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048768+0000","last_change":"2024-02-07T13:09:57.050107+0000","last_active":"2024-02-07T13:10:27.048768+0000","last_peered":"2024-02-07T13:10:27.048768+0000","last_clean":"2024-02-07T13:10:27.048768+0000","last_became_active":"2024-02-07T13:09:57.049309+0000","last_became_peered":"2024-02-07T13:09:57.049309+0000","last_unstale":"2024-02-07T13:10:27.048768+0000","last_undegraded":"2024-02-07T13:10:27.048768+0000","last_fullsized":"2024-02-07T13:10:27.048768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:18:09.378380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"31'10","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048992+0000","last_change":"2024-02-07T13:09:59.059297+0000","last_active":"2024-02-07T13:10:27.048992+0000","last_peered":"2024-02-07T13:10:27.048992+0000","last_clean":"2024-02-07T13:10:27.048992+0000","last_became_active":"2024-02-07T13:09:59.058474+0000","last_became_peered":"2024-02-07T13:09:59.058474+0000","last_unstale":"2024-02-07T13:10:27.048992+0000","last_undegraded":"2024-02-07T13:10:27.048992+0000","last_fullsized":"2024-02-07T13:10:27.048992+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:59:10.990587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046697+0000","last_change":"2024-02-07T13:10:01.069709+0000","last_active":"2024-02-07T13:10:27.046697+0000","last_peered":"2024-02-07T13:10:27.046697+0000","last_clean":"2024-02-07T13:10:27.046697+0000","last_became_active":"2024-02-07T13:10:01.069398+0000","last_became_peered":"2024-02-07T13:10:01.069398+0000","last_unstale":"2024-02-07T13:10:27.046697+0000","last_undegraded":"2024-02-07T13:10:27.046697+0000","last_fullsized":"2024-02-07T13:10:27.046697+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:13:01.382113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046901+0000","last_change":"2024-02-07T13:10:03.085552+0000","last_active":"2024-02-07T13:10:27.046901+0000","last_peered":"2024-02-07T13:10:27.046901+0000","last_clean":"2024-02-07T13:10:27.046901+0000","last_became_active":"2024-02-07T13:10:03.085225+0000","last_became_peered":"2024-02-07T13:10:03.085225+0000","last_unstale":"2024-02-07T13:10:27.046901+0000","last_undegraded":"2024-02-07T13:10:27.046901+0000","last_fullsized":"2024-02-07T13:10:27.046901+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:55:41.888615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049477+0000","last_change":"2024-02-07T13:09:57.046664+0000","last_active":"2024-02-07T13:10:27.049477+0000","last_peered":"2024-02-07T13:10:27.049477+0000","last_clean":"2024-02-07T13:10:27.049477+0000","last_became_active":"2024-02-07T13:09:57.045882+0000","last_became_peered":"2024-02-07T13:09:57.045882+0000","last_unstale":"2024-02-07T13:10:27.049477+0000","last_undegraded":"2024-02-07T13:10:27.049477+0000","last_fullsized":"2024-02-07T13:10:27.049477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:48:29.700282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"31'15","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049365+0000","last_change":"2024-02-07T13:09:59.059244+0000","last_active":"2024-02-07T13:10:27.049365+0000","last_peered":"2024-02-07T13:10:27.049365+0000","last_clean":"2024-02-07T13:10:27.049365+0000","last_became_active":"2024-02-07T13:09:59.058912+0000","last_became_peered":"2024-02-07T13:09:59.058912+0000","last_unstale":"2024-02-07T13:10:27.049365+0000","last_undegraded":"2024-02-07T13:10:27.049365+0000","last_fullsized":"2024-02-07T13:10:27.049365+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:05:23.210608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047584+0000","last_change":"2024-02-07T13:10:01.068095+0000","last_active":"2024-02-07T13:10:27.047584+0000","last_peered":"2024-02-07T13:10:27.047584+0000","last_clean":"2024-02-07T13:10:27.047584+0000","last_became_active":"2024-02-07T13:10:01.067769+0000","last_became_peered":"2024-02-07T13:10:01.067769+0000","last_unstale":"2024-02-07T13:10:27.047584+0000","last_undegraded":"2024-02-07T13:10:27.047584+0000","last_fullsized":"2024-02-07T13:10:27.047584+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:54:13.238916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411345,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708840,"statfs":{"total":95990841344,"available":95957852160,"internally_reserved":0,"allocated":1060864,"data_stored":703110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 7 13:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.307}]},{"osd":1,"last update":"Wed Feb 7 13:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.32700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738395,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708844,"statfs":{"total":95990841344,"available":95957856256,"internally_reserved":0,"allocated":1060864,"data_stored":703110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 7 13:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.33300000000000002}]},{"osd":2,"last update":"Wed Feb 7 13:10:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.49199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708844,"statfs":{"total":95990841344,"available":95957856256,"internally_reserved":0,"allocated":1060864,"data_stored":703110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 7 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.31}]},{"osd":2,"last update":"Wed Feb 7 13:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.37},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.38300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-07T13:10:56.208 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph pg dump --format=json 2024-02-07T13:10:56.448 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:10:57.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:56 smithi159 ceph-mon[27461]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:57.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:56 smithi159 ceph-mon[27461]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:10:57.283 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:56 smithi067 ceph-mon[25698]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:57.283 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:56 smithi067 ceph-mon[25698]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:10:57.368 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:56 smithi191 ceph-mon[27623]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:57.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:56 smithi191 ceph-mon[27623]: from='client.14535 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:10:57.905 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:57 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:10:58.077 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:57 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:10:58.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:57 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:10:58.379 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:10:58.381 INFO:teuthology.orchestra.run.smithi067.stderr:dumped all 2024-02-07T13:10:58.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:58 smithi159 ceph-mon[27461]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:59.143 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:58 smithi067 ceph-mon[25698]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:59.144 INFO:teuthology.orchestra.run.smithi067.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-02-07T13:10:57.064189+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595908,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":974,"num_read_kb":821,"num_write":697,"num_write_kb":4546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":673,"ondisk_log_size":673,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":96640,"kb_used_data":3108,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":281126528,"statfs":{"total":287972524032,"available":287873564672,"internally_reserved":0,"allocated":3182592,"data_stored":2109330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002772"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047340+0000","last_change":"2024-02-07T13:10:03.088209+0000","last_active":"2024-02-07T13:10:27.047340+0000","last_peered":"2024-02-07T13:10:27.047340+0000","last_clean":"2024-02-07T13:10:27.047340+0000","last_became_active":"2024-02-07T13:10:03.087867+0000","last_became_peered":"2024-02-07T13:10:03.087867+0000","last_unstale":"2024-02-07T13:10:27.047340+0000","last_undegraded":"2024-02-07T13:10:27.047340+0000","last_fullsized":"2024-02-07T13:10:27.047340+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:27:05.993847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047025+0000","last_change":"2024-02-07T13:10:01.069894+0000","last_active":"2024-02-07T13:10:27.047025+0000","last_peered":"2024-02-07T13:10:27.047025+0000","last_clean":"2024-02-07T13:10:27.047025+0000","last_became_active":"2024-02-07T13:10:01.069609+0000","last_became_peered":"2024-02-07T13:10:01.069609+0000","last_unstale":"2024-02-07T13:10:27.047025+0000","last_undegraded":"2024-02-07T13:10:27.047025+0000","last_fullsized":"2024-02-07T13:10:27.047025+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:56:48.582558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049521+0000","last_change":"2024-02-07T13:09:57.046326+0000","last_active":"2024-02-07T13:10:27.049521+0000","last_peered":"2024-02-07T13:10:27.049521+0000","last_clean":"2024-02-07T13:10:27.049521+0000","last_became_active":"2024-02-07T13:09:57.045877+0000","last_became_peered":"2024-02-07T13:09:57.045877+0000","last_unstale":"2024-02-07T13:10:27.049521+0000","last_undegraded":"2024-02-07T13:10:27.049521+0000","last_fullsized":"2024-02-07T13:10:27.049521+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:07:16.828704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"31'10","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049402+0000","last_change":"2024-02-07T13:09:59.060952+0000","last_active":"2024-02-07T13:10:27.049402+0000","last_peered":"2024-02-07T13:10:27.049402+0000","last_clean":"2024-02-07T13:10:27.049402+0000","last_became_active":"2024-02-07T13:09:59.060387+0000","last_became_peered":"2024-02-07T13:09:59.060387+0000","last_unstale":"2024-02-07T13:10:27.049402+0000","last_undegraded":"2024-02-07T13:10:27.049402+0000","last_fullsized":"2024-02-07T13:10:27.049402+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:14:57.814494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046053+0000","last_change":"2024-02-07T13:10:03.083060+0000","last_active":"2024-02-07T13:10:27.046053+0000","last_peered":"2024-02-07T13:10:27.046053+0000","last_clean":"2024-02-07T13:10:27.046053+0000","last_became_active":"2024-02-07T13:10:03.082572+0000","last_became_peered":"2024-02-07T13:10:03.082572+0000","last_unstale":"2024-02-07T13:10:27.046053+0000","last_undegraded":"2024-02-07T13:10:27.046053+0000","last_fullsized":"2024-02-07T13:10:27.046053+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:40:35.919055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047189+0000","last_change":"2024-02-07T13:10:01.072342+0000","last_active":"2024-02-07T13:10:27.047189+0000","last_peered":"2024-02-07T13:10:27.047189+0000","last_clean":"2024-02-07T13:10:27.047189+0000","last_became_active":"2024-02-07T13:10:01.071845+0000","last_became_peered":"2024-02-07T13:10:01.071845+0000","last_unstale":"2024-02-07T13:10:27.047189+0000","last_undegraded":"2024-02-07T13:10:27.047189+0000","last_fullsized":"2024-02-07T13:10:27.047189+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:24:00.754390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049566+0000","last_change":"2024-02-07T13:09:57.046796+0000","last_active":"2024-02-07T13:10:27.049566+0000","last_peered":"2024-02-07T13:10:27.049566+0000","last_clean":"2024-02-07T13:10:27.049566+0000","last_became_active":"2024-02-07T13:09:57.046159+0000","last_became_peered":"2024-02-07T13:09:57.046159+0000","last_unstale":"2024-02-07T13:10:27.049566+0000","last_undegraded":"2024-02-07T13:10:27.049566+0000","last_fullsized":"2024-02-07T13:10:27.049566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:08:29.765095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"31'11","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049177+0000","last_change":"2024-02-07T13:09:59.060218+0000","last_active":"2024-02-07T13:10:27.049177+0000","last_peered":"2024-02-07T13:10:27.049177+0000","last_clean":"2024-02-07T13:10:27.049177+0000","last_became_active":"2024-02-07T13:09:59.059734+0000","last_became_peered":"2024-02-07T13:09:59.059734+0000","last_unstale":"2024-02-07T13:10:27.049177+0000","last_undegraded":"2024-02-07T13:10:27.049177+0000","last_fullsized":"2024-02-07T13:10:27.049177+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:42:27.742686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047155+0000","last_change":"2024-02-07T13:10:03.084542+0000","last_active":"2024-02-07T13:10:27.047155+0000","last_peered":"2024-02-07T13:10:27.047155+0000","last_clean":"2024-02-07T13:10:27.047155+0000","last_became_active":"2024-02-07T13:10:03.084107+0000","last_became_peered":"2024-02-07T13:10:03.084107+0000","last_unstale":"2024-02-07T13:10:27.047155+0000","last_undegraded":"2024-02-07T13:10:27.047155+0000","last_fullsized":"2024-02-07T13:10:27.047155+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:34:52.337365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048073+0000","last_change":"2024-02-07T13:10:01.072094+0000","last_active":"2024-02-07T13:10:27.048073+0000","last_peered":"2024-02-07T13:10:27.048073+0000","last_clean":"2024-02-07T13:10:27.048073+0000","last_became_active":"2024-02-07T13:10:01.071629+0000","last_became_peered":"2024-02-07T13:10:01.071629+0000","last_unstale":"2024-02-07T13:10:27.048073+0000","last_undegraded":"2024-02-07T13:10:27.048073+0000","last_fullsized":"2024-02-07T13:10:27.048073+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:47:39.945764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049620+0000","last_change":"2024-02-07T13:09:57.050009+0000","last_active":"2024-02-07T13:10:27.049620+0000","last_peered":"2024-02-07T13:10:27.049620+0000","last_clean":"2024-02-07T13:10:27.049620+0000","last_became_active":"2024-02-07T13:09:57.048095+0000","last_became_peered":"2024-02-07T13:09:57.048095+0000","last_unstale":"2024-02-07T13:10:27.049620+0000","last_undegraded":"2024-02-07T13:10:27.049620+0000","last_fullsized":"2024-02-07T13:10:27.049620+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:30:07.938219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"31'15","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049081+0000","last_change":"2024-02-07T13:09:59.057441+0000","last_active":"2024-02-07T13:10:27.049081+0000","last_peered":"2024-02-07T13:10:27.049081+0000","last_clean":"2024-02-07T13:10:27.049081+0000","last_became_active":"2024-02-07T13:09:59.057080+0000","last_became_peered":"2024-02-07T13:09:59.057080+0000","last_unstale":"2024-02-07T13:10:27.049081+0000","last_undegraded":"2024-02-07T13:10:27.049081+0000","last_fullsized":"2024-02-07T13:10:27.049081+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:24:12.244346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047800+0000","last_change":"2024-02-07T13:10:03.085119+0000","last_active":"2024-02-07T13:10:27.047800+0000","last_peered":"2024-02-07T13:10:27.047800+0000","last_clean":"2024-02-07T13:10:27.047800+0000","last_became_active":"2024-02-07T13:10:03.084349+0000","last_became_peered":"2024-02-07T13:10:03.084349+0000","last_unstale":"2024-02-07T13:10:27.047800+0000","last_undegraded":"2024-02-07T13:10:27.047800+0000","last_fullsized":"2024-02-07T13:10:27.047800+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:36:57.272469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047394+0000","last_change":"2024-02-07T13:10:01.070229+0000","last_active":"2024-02-07T13:10:27.047394+0000","last_peered":"2024-02-07T13:10:27.047394+0000","last_clean":"2024-02-07T13:10:27.047394+0000","last_became_active":"2024-02-07T13:10:01.069778+0000","last_became_peered":"2024-02-07T13:10:01.069778+0000","last_unstale":"2024-02-07T13:10:27.047394+0000","last_undegraded":"2024-02-07T13:10:27.047394+0000","last_fullsized":"2024-02-07T13:10:27.047394+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:24:16.008928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048388+0000","last_change":"2024-02-07T13:09:57.049301+0000","last_active":"2024-02-07T13:10:27.048388+0000","last_peered":"2024-02-07T13:10:27.048388+0000","last_clean":"2024-02-07T13:10:27.048388+0000","last_became_active":"2024-02-07T13:09:57.047789+0000","last_became_peered":"2024-02-07T13:09:57.047789+0000","last_unstale":"2024-02-07T13:10:27.048388+0000","last_undegraded":"2024-02-07T13:10:27.048388+0000","last_fullsized":"2024-02-07T13:10:27.048388+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:58:25.388168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"31'12","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049553+0000","last_change":"2024-02-07T13:09:59.057769+0000","last_active":"2024-02-07T13:10:27.049553+0000","last_peered":"2024-02-07T13:10:27.049553+0000","last_clean":"2024-02-07T13:10:27.049553+0000","last_became_active":"2024-02-07T13:09:59.057482+0000","last_became_peered":"2024-02-07T13:09:59.057482+0000","last_unstale":"2024-02-07T13:10:27.049553+0000","last_undegraded":"2024-02-07T13:10:27.049553+0000","last_fullsized":"2024-02-07T13:10:27.049553+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:34:29.980756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048643+0000","last_change":"2024-02-07T13:10:03.084807+0000","last_active":"2024-02-07T13:10:27.048643+0000","last_peered":"2024-02-07T13:10:27.048643+0000","last_clean":"2024-02-07T13:10:27.048643+0000","last_became_active":"2024-02-07T13:10:03.084381+0000","last_became_peered":"2024-02-07T13:10:03.084381+0000","last_unstale":"2024-02-07T13:10:27.048643+0000","last_undegraded":"2024-02-07T13:10:27.048643+0000","last_fullsized":"2024-02-07T13:10:27.048643+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:53:40.147298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048615+0000","last_change":"2024-02-07T13:10:01.067690+0000","last_active":"2024-02-07T13:10:27.048615+0000","last_peered":"2024-02-07T13:10:27.048615+0000","last_clean":"2024-02-07T13:10:27.048615+0000","last_became_active":"2024-02-07T13:10:01.067409+0000","last_became_peered":"2024-02-07T13:10:01.067409+0000","last_unstale":"2024-02-07T13:10:27.048615+0000","last_undegraded":"2024-02-07T13:10:27.048615+0000","last_fullsized":"2024-02-07T13:10:27.048615+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:38:49.161734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048732+0000","last_change":"2024-02-07T13:09:57.047606+0000","last_active":"2024-02-07T13:10:27.048732+0000","last_peered":"2024-02-07T13:10:27.048732+0000","last_clean":"2024-02-07T13:10:27.048732+0000","last_became_active":"2024-02-07T13:09:57.046293+0000","last_became_peered":"2024-02-07T13:09:57.046293+0000","last_unstale":"2024-02-07T13:10:27.048732+0000","last_undegraded":"2024-02-07T13:10:27.048732+0000","last_fullsized":"2024-02-07T13:10:27.048732+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:54:57.181133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048008+0000","last_change":"2024-02-07T13:09:59.062887+0000","last_active":"2024-02-07T13:10:27.048008+0000","last_peered":"2024-02-07T13:10:27.048008+0000","last_clean":"2024-02-07T13:10:27.048008+0000","last_became_active":"2024-02-07T13:09:59.062488+0000","last_became_peered":"2024-02-07T13:09:59.062488+0000","last_unstale":"2024-02-07T13:10:27.048008+0000","last_undegraded":"2024-02-07T13:10:27.048008+0000","last_fullsized":"2024-02-07T13:10:27.048008+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:23:19.523618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048649+0000","last_change":"2024-02-07T13:10:03.088567+0000","last_active":"2024-02-07T13:10:27.048649+0000","last_peered":"2024-02-07T13:10:27.048649+0000","last_clean":"2024-02-07T13:10:27.048649+0000","last_became_active":"2024-02-07T13:10:03.088241+0000","last_became_peered":"2024-02-07T13:10:03.088241+0000","last_unstale":"2024-02-07T13:10:27.048649+0000","last_undegraded":"2024-02-07T13:10:27.048649+0000","last_fullsized":"2024-02-07T13:10:27.048649+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:02:06.349020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048237+0000","last_change":"2024-02-07T13:10:01.070847+0000","last_active":"2024-02-07T13:10:27.048237+0000","last_peered":"2024-02-07T13:10:27.048237+0000","last_clean":"2024-02-07T13:10:27.048237+0000","last_became_active":"2024-02-07T13:10:01.070110+0000","last_became_peered":"2024-02-07T13:10:01.070110+0000","last_unstale":"2024-02-07T13:10:27.048237+0000","last_undegraded":"2024-02-07T13:10:27.048237+0000","last_fullsized":"2024-02-07T13:10:27.048237+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:59:26.226974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047582+0000","last_change":"2024-02-07T13:09:57.052334+0000","last_active":"2024-02-07T13:10:27.047582+0000","last_peered":"2024-02-07T13:10:27.047582+0000","last_clean":"2024-02-07T13:10:27.047582+0000","last_became_active":"2024-02-07T13:09:57.052004+0000","last_became_peered":"2024-02-07T13:09:57.052004+0000","last_unstale":"2024-02-07T13:10:27.047582+0000","last_undegraded":"2024-02-07T13:10:27.047582+0000","last_fullsized":"2024-02-07T13:10:27.047582+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:46:25.083833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"31'5","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048203+0000","last_change":"2024-02-07T13:09:59.058470+0000","last_active":"2024-02-07T13:10:27.048203+0000","last_peered":"2024-02-07T13:10:27.048203+0000","last_clean":"2024-02-07T13:10:27.048203+0000","last_became_active":"2024-02-07T13:09:59.058073+0000","last_became_peered":"2024-02-07T13:09:59.058073+0000","last_unstale":"2024-02-07T13:10:27.048203+0000","last_undegraded":"2024-02-07T13:10:27.048203+0000","last_fullsized":"2024-02-07T13:10:27.048203+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:53:46.824292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048846+0000","last_change":"2024-02-07T13:10:03.089462+0000","last_active":"2024-02-07T13:10:27.048846+0000","last_peered":"2024-02-07T13:10:27.048846+0000","last_clean":"2024-02-07T13:10:27.048846+0000","last_became_active":"2024-02-07T13:10:03.089156+0000","last_became_peered":"2024-02-07T13:10:03.089156+0000","last_unstale":"2024-02-07T13:10:27.048846+0000","last_undegraded":"2024-02-07T13:10:27.048846+0000","last_fullsized":"2024-02-07T13:10:27.048846+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:04:35.809606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'8","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099723+0000","last_change":"2024-02-07T13:10:01.071307+0000","last_active":"2024-02-07T13:10:47.099723+0000","last_peered":"2024-02-07T13:10:47.099723+0000","last_clean":"2024-02-07T13:10:47.099723+0000","last_became_active":"2024-02-07T13:10:01.070650+0000","last_became_peered":"2024-02-07T13:10:01.070650+0000","last_unstale":"2024-02-07T13:10:47.099723+0000","last_undegraded":"2024-02-07T13:10:47.099723+0000","last_fullsized":"2024-02-07T13:10:47.099723+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:42:16.500508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"24'1","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046247+0000","last_change":"2024-02-07T13:09:57.046324+0000","last_active":"2024-02-07T13:10:27.046247+0000","last_peered":"2024-02-07T13:10:27.046247+0000","last_clean":"2024-02-07T13:10:27.046247+0000","last_became_active":"2024-02-07T13:09:57.045892+0000","last_became_peered":"2024-02-07T13:09:57.045892+0000","last_unstale":"2024-02-07T13:10:27.046247+0000","last_undegraded":"2024-02-07T13:10:27.046247+0000","last_fullsized":"2024-02-07T13:10:27.046247+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:48:06.526469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.045782+0000","last_change":"2024-02-07T13:09:59.059070+0000","last_active":"2024-02-07T13:10:27.045782+0000","last_peered":"2024-02-07T13:10:27.045782+0000","last_clean":"2024-02-07T13:10:27.045782+0000","last_became_active":"2024-02-07T13:09:59.058480+0000","last_became_peered":"2024-02-07T13:09:59.058480+0000","last_unstale":"2024-02-07T13:10:27.045782+0000","last_undegraded":"2024-02-07T13:10:27.045782+0000","last_fullsized":"2024-02-07T13:10:27.045782+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:56:59.792412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048655+0000","last_change":"2024-02-07T13:10:03.084631+0000","last_active":"2024-02-07T13:10:27.048655+0000","last_peered":"2024-02-07T13:10:27.048655+0000","last_clean":"2024-02-07T13:10:27.048655+0000","last_became_active":"2024-02-07T13:10:03.083922+0000","last_became_peered":"2024-02-07T13:10:03.083922+0000","last_unstale":"2024-02-07T13:10:27.048655+0000","last_undegraded":"2024-02-07T13:10:27.048655+0000","last_fullsized":"2024-02-07T13:10:27.048655+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:16:44.069054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049144+0000","last_change":"2024-02-07T13:10:01.070082+0000","last_active":"2024-02-07T13:10:27.049144+0000","last_peered":"2024-02-07T13:10:27.049144+0000","last_clean":"2024-02-07T13:10:27.049144+0000","last_became_active":"2024-02-07T13:10:01.069636+0000","last_became_peered":"2024-02-07T13:10:01.069636+0000","last_unstale":"2024-02-07T13:10:27.049144+0000","last_undegraded":"2024-02-07T13:10:27.049144+0000","last_fullsized":"2024-02-07T13:10:27.049144+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:33:00.277242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047217+0000","last_change":"2024-02-07T13:09:57.046738+0000","last_active":"2024-02-07T13:10:27.047217+0000","last_peered":"2024-02-07T13:10:27.047217+0000","last_clean":"2024-02-07T13:10:27.047217+0000","last_became_active":"2024-02-07T13:09:57.046394+0000","last_became_peered":"2024-02-07T13:09:57.046394+0000","last_unstale":"2024-02-07T13:10:27.047217+0000","last_undegraded":"2024-02-07T13:10:27.047217+0000","last_fullsized":"2024-02-07T13:10:27.047217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:58:59.957430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"31'15","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.045902+0000","last_change":"2024-02-07T13:09:59.060184+0000","last_active":"2024-02-07T13:10:27.045902+0000","last_peered":"2024-02-07T13:10:27.045902+0000","last_clean":"2024-02-07T13:10:27.045902+0000","last_became_active":"2024-02-07T13:09:59.059858+0000","last_became_peered":"2024-02-07T13:09:59.059858+0000","last_unstale":"2024-02-07T13:10:27.045902+0000","last_undegraded":"2024-02-07T13:10:27.045902+0000","last_fullsized":"2024-02-07T13:10:27.045902+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:44:25.690571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047451+0000","last_change":"2024-02-07T13:10:03.083358+0000","last_active":"2024-02-07T13:10:27.047451+0000","last_peered":"2024-02-07T13:10:27.047451+0000","last_clean":"2024-02-07T13:10:27.047451+0000","last_became_active":"2024-02-07T13:10:03.082915+0000","last_became_peered":"2024-02-07T13:10:03.082915+0000","last_unstale":"2024-02-07T13:10:27.047451+0000","last_undegraded":"2024-02-07T13:10:27.047451+0000","last_fullsized":"2024-02-07T13:10:27.047451+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:53:06.675693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.045799+0000","last_change":"2024-02-07T13:10:01.069233+0000","last_active":"2024-02-07T13:10:27.045799+0000","last_peered":"2024-02-07T13:10:27.045799+0000","last_clean":"2024-02-07T13:10:27.045799+0000","last_became_active":"2024-02-07T13:10:01.068869+0000","last_became_peered":"2024-02-07T13:10:01.068869+0000","last_unstale":"2024-02-07T13:10:27.045799+0000","last_undegraded":"2024-02-07T13:10:27.045799+0000","last_fullsized":"2024-02-07T13:10:27.045799+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:37:24.027498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049664+0000","last_change":"2024-02-07T13:09:57.051664+0000","last_active":"2024-02-07T13:10:27.049664+0000","last_peered":"2024-02-07T13:10:27.049664+0000","last_clean":"2024-02-07T13:10:27.049664+0000","last_became_active":"2024-02-07T13:09:57.051234+0000","last_became_peered":"2024-02-07T13:09:57.051234+0000","last_unstale":"2024-02-07T13:10:27.049664+0000","last_undegraded":"2024-02-07T13:10:27.049664+0000","last_fullsized":"2024-02-07T13:10:27.049664+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:47:30.416264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.050054+0000","last_change":"2024-02-07T13:09:59.059201+0000","last_active":"2024-02-07T13:10:27.050054+0000","last_peered":"2024-02-07T13:10:27.050054+0000","last_clean":"2024-02-07T13:10:27.050054+0000","last_became_active":"2024-02-07T13:09:59.058685+0000","last_became_peered":"2024-02-07T13:09:59.058685+0000","last_unstale":"2024-02-07T13:10:27.050054+0000","last_undegraded":"2024-02-07T13:10:27.050054+0000","last_fullsized":"2024-02-07T13:10:27.050054+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T01:07:06.590953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047225+0000","last_change":"2024-02-07T13:10:03.083280+0000","last_active":"2024-02-07T13:10:27.047225+0000","last_peered":"2024-02-07T13:10:27.047225+0000","last_clean":"2024-02-07T13:10:27.047225+0000","last_became_active":"2024-02-07T13:10:03.082779+0000","last_became_peered":"2024-02-07T13:10:03.082779+0000","last_unstale":"2024-02-07T13:10:27.047225+0000","last_undegraded":"2024-02-07T13:10:27.047225+0000","last_fullsized":"2024-02-07T13:10:27.047225+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:54:23.210248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046172+0000","last_change":"2024-02-07T13:10:01.068136+0000","last_active":"2024-02-07T13:10:27.046172+0000","last_peered":"2024-02-07T13:10:27.046172+0000","last_clean":"2024-02-07T13:10:27.046172+0000","last_became_active":"2024-02-07T13:10:01.067735+0000","last_became_peered":"2024-02-07T13:10:01.067735+0000","last_unstale":"2024-02-07T13:10:27.046172+0000","last_undegraded":"2024-02-07T13:10:27.046172+0000","last_fullsized":"2024-02-07T13:10:27.046172+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:13:08.582408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049841+0000","last_change":"2024-02-07T13:09:57.051248+0000","last_active":"2024-02-07T13:10:27.049841+0000","last_peered":"2024-02-07T13:10:27.049841+0000","last_clean":"2024-02-07T13:10:27.049841+0000","last_became_active":"2024-02-07T13:09:57.050765+0000","last_became_peered":"2024-02-07T13:09:57.050765+0000","last_unstale":"2024-02-07T13:10:27.049841+0000","last_undegraded":"2024-02-07T13:10:27.049841+0000","last_fullsized":"2024-02-07T13:10:27.049841+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:16:31.053960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"31'6","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049677+0000","last_change":"2024-02-07T13:09:59.060703+0000","last_active":"2024-02-07T13:10:27.049677+0000","last_peered":"2024-02-07T13:10:27.049677+0000","last_clean":"2024-02-07T13:10:27.049677+0000","last_became_active":"2024-02-07T13:09:59.060407+0000","last_became_peered":"2024-02-07T13:09:59.060407+0000","last_unstale":"2024-02-07T13:10:27.049677+0000","last_undegraded":"2024-02-07T13:10:27.049677+0000","last_fullsized":"2024-02-07T13:10:27.049677+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:35:13.749039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047371+0000","last_change":"2024-02-07T13:10:03.084806+0000","last_active":"2024-02-07T13:10:27.047371+0000","last_peered":"2024-02-07T13:10:27.047371+0000","last_clean":"2024-02-07T13:10:27.047371+0000","last_became_active":"2024-02-07T13:10:03.084461+0000","last_became_peered":"2024-02-07T13:10:03.084461+0000","last_unstale":"2024-02-07T13:10:27.047371+0000","last_undegraded":"2024-02-07T13:10:27.047371+0000","last_fullsized":"2024-02-07T13:10:27.047371+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:07:16.594572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048153+0000","last_change":"2024-02-07T13:10:01.069477+0000","last_active":"2024-02-07T13:10:27.048153+0000","last_peered":"2024-02-07T13:10:27.048153+0000","last_clean":"2024-02-07T13:10:27.048153+0000","last_became_active":"2024-02-07T13:10:01.069137+0000","last_became_peered":"2024-02-07T13:10:01.069137+0000","last_unstale":"2024-02-07T13:10:27.048153+0000","last_undegraded":"2024-02-07T13:10:27.048153+0000","last_fullsized":"2024-02-07T13:10:27.048153+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:59:38.130120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049514+0000","last_change":"2024-02-07T13:09:57.049940+0000","last_active":"2024-02-07T13:10:27.049514+0000","last_peered":"2024-02-07T13:10:27.049514+0000","last_clean":"2024-02-07T13:10:27.049514+0000","last_became_active":"2024-02-07T13:09:57.048046+0000","last_became_peered":"2024-02-07T13:09:57.048046+0000","last_unstale":"2024-02-07T13:10:27.049514+0000","last_undegraded":"2024-02-07T13:10:27.049514+0000","last_fullsized":"2024-02-07T13:10:27.049514+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:42:39.845051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"31'10","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049223+0000","last_change":"2024-02-07T13:09:59.057613+0000","last_active":"2024-02-07T13:10:27.049223+0000","last_peered":"2024-02-07T13:10:27.049223+0000","last_clean":"2024-02-07T13:10:27.049223+0000","last_became_active":"2024-02-07T13:09:59.057365+0000","last_became_peered":"2024-02-07T13:09:59.057365+0000","last_unstale":"2024-02-07T13:10:27.049223+0000","last_undegraded":"2024-02-07T13:10:27.049223+0000","last_fullsized":"2024-02-07T13:10:27.049223+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:18:45.692893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047345+0000","last_change":"2024-02-07T13:10:03.084409+0000","last_active":"2024-02-07T13:10:27.047345+0000","last_peered":"2024-02-07T13:10:27.047345+0000","last_clean":"2024-02-07T13:10:27.047345+0000","last_became_active":"2024-02-07T13:10:03.084127+0000","last_became_peered":"2024-02-07T13:10:03.084127+0000","last_unstale":"2024-02-07T13:10:27.047345+0000","last_undegraded":"2024-02-07T13:10:27.047345+0000","last_fullsized":"2024-02-07T13:10:27.047345+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:03:02.620255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046379+0000","last_change":"2024-02-07T13:10:01.067967+0000","last_active":"2024-02-07T13:10:27.046379+0000","last_peered":"2024-02-07T13:10:27.046379+0000","last_clean":"2024-02-07T13:10:27.046379+0000","last_became_active":"2024-02-07T13:10:01.067674+0000","last_became_peered":"2024-02-07T13:10:01.067674+0000","last_unstale":"2024-02-07T13:10:27.046379+0000","last_undegraded":"2024-02-07T13:10:27.046379+0000","last_fullsized":"2024-02-07T13:10:27.046379+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:16:26.922137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048990+0000","last_change":"2024-02-07T13:09:57.051648+0000","last_active":"2024-02-07T13:10:27.048990+0000","last_peered":"2024-02-07T13:10:27.048990+0000","last_clean":"2024-02-07T13:10:27.048990+0000","last_became_active":"2024-02-07T13:09:57.051316+0000","last_became_peered":"2024-02-07T13:09:57.051316+0000","last_unstale":"2024-02-07T13:10:27.048990+0000","last_undegraded":"2024-02-07T13:10:27.048990+0000","last_fullsized":"2024-02-07T13:10:27.048990+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:49:46.881103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049324+0000","last_change":"2024-02-07T13:09:59.058450+0000","last_active":"2024-02-07T13:10:27.049324+0000","last_peered":"2024-02-07T13:10:27.049324+0000","last_clean":"2024-02-07T13:10:27.049324+0000","last_became_active":"2024-02-07T13:09:59.058181+0000","last_became_peered":"2024-02-07T13:09:59.058181+0000","last_unstale":"2024-02-07T13:10:27.049324+0000","last_undegraded":"2024-02-07T13:10:27.049324+0000","last_fullsized":"2024-02-07T13:10:27.049324+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:52:07.701473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047565+0000","last_change":"2024-02-07T13:10:03.088471+0000","last_active":"2024-02-07T13:10:27.047565+0000","last_peered":"2024-02-07T13:10:27.047565+0000","last_clean":"2024-02-07T13:10:27.047565+0000","last_became_active":"2024-02-07T13:10:03.088045+0000","last_became_peered":"2024-02-07T13:10:03.088045+0000","last_unstale":"2024-02-07T13:10:27.047565+0000","last_undegraded":"2024-02-07T13:10:27.047565+0000","last_fullsized":"2024-02-07T13:10:27.047565+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:09:33.150941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"31'8","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099678+0000","last_change":"2024-02-07T13:10:01.067694+0000","last_active":"2024-02-07T13:10:47.099678+0000","last_peered":"2024-02-07T13:10:47.099678+0000","last_clean":"2024-02-07T13:10:47.099678+0000","last_became_active":"2024-02-07T13:10:01.067382+0000","last_became_peered":"2024-02-07T13:10:01.067382+0000","last_unstale":"2024-02-07T13:10:47.099678+0000","last_undegraded":"2024-02-07T13:10:47.099678+0000","last_fullsized":"2024-02-07T13:10:47.099678+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:46:40.939112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048017+0000","last_change":"2024-02-07T13:09:57.049270+0000","last_active":"2024-02-07T13:10:27.048017+0000","last_peered":"2024-02-07T13:10:27.048017+0000","last_clean":"2024-02-07T13:10:27.048017+0000","last_became_active":"2024-02-07T13:09:57.048114+0000","last_became_peered":"2024-02-07T13:09:57.048114+0000","last_unstale":"2024-02-07T13:10:27.048017+0000","last_undegraded":"2024-02-07T13:10:27.048017+0000","last_fullsized":"2024-02-07T13:10:27.048017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:12:14.607256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046592+0000","last_change":"2024-02-07T13:09:59.058720+0000","last_active":"2024-02-07T13:10:27.046592+0000","last_peered":"2024-02-07T13:10:27.046592+0000","last_clean":"2024-02-07T13:10:27.046592+0000","last_became_active":"2024-02-07T13:09:59.058371+0000","last_became_peered":"2024-02-07T13:09:59.058371+0000","last_unstale":"2024-02-07T13:10:27.046592+0000","last_undegraded":"2024-02-07T13:10:27.046592+0000","last_fullsized":"2024-02-07T13:10:27.046592+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:34:08.111432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048353+0000","last_change":"2024-02-07T13:10:03.085577+0000","last_active":"2024-02-07T13:10:27.048353+0000","last_peered":"2024-02-07T13:10:27.048353+0000","last_clean":"2024-02-07T13:10:27.048353+0000","last_became_active":"2024-02-07T13:10:03.085269+0000","last_became_peered":"2024-02-07T13:10:03.085269+0000","last_unstale":"2024-02-07T13:10:27.048353+0000","last_undegraded":"2024-02-07T13:10:27.048353+0000","last_fullsized":"2024-02-07T13:10:27.048353+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:39:05.976236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"31'8","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099699+0000","last_change":"2024-02-07T13:10:01.070016+0000","last_active":"2024-02-07T13:10:47.099699+0000","last_peered":"2024-02-07T13:10:47.099699+0000","last_clean":"2024-02-07T13:10:47.099699+0000","last_became_active":"2024-02-07T13:10:01.069713+0000","last_became_peered":"2024-02-07T13:10:01.069713+0000","last_unstale":"2024-02-07T13:10:47.099699+0000","last_undegraded":"2024-02-07T13:10:47.099699+0000","last_fullsized":"2024-02-07T13:10:47.099699+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:34:21.958265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"24'1","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047631+0000","last_change":"2024-02-07T13:09:57.048109+0000","last_active":"2024-02-07T13:10:27.047631+0000","last_peered":"2024-02-07T13:10:27.047631+0000","last_clean":"2024-02-07T13:10:27.047631+0000","last_became_active":"2024-02-07T13:09:57.046824+0000","last_became_peered":"2024-02-07T13:09:57.046824+0000","last_unstale":"2024-02-07T13:10:27.047631+0000","last_undegraded":"2024-02-07T13:10:27.047631+0000","last_fullsized":"2024-02-07T13:10:27.047631+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:01:09.822849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"31'11","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048784+0000","last_change":"2024-02-07T13:09:59.056921+0000","last_active":"2024-02-07T13:10:27.048784+0000","last_peered":"2024-02-07T13:10:27.048784+0000","last_clean":"2024-02-07T13:10:27.048784+0000","last_became_active":"2024-02-07T13:09:59.056619+0000","last_became_peered":"2024-02-07T13:09:59.056619+0000","last_unstale":"2024-02-07T13:10:27.048784+0000","last_undegraded":"2024-02-07T13:10:27.048784+0000","last_fullsized":"2024-02-07T13:10:27.048784+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:40:39.394443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049465+0000","last_change":"2024-02-07T13:10:03.084939+0000","last_active":"2024-02-07T13:10:27.049465+0000","last_peered":"2024-02-07T13:10:27.049465+0000","last_clean":"2024-02-07T13:10:27.049465+0000","last_became_active":"2024-02-07T13:10:03.083993+0000","last_became_peered":"2024-02-07T13:10:03.083993+0000","last_unstale":"2024-02-07T13:10:27.049465+0000","last_undegraded":"2024-02-07T13:10:27.049465+0000","last_fullsized":"2024-02-07T13:10:27.049465+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:50:15.026004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049086+0000","last_change":"2024-02-07T13:10:01.071023+0000","last_active":"2024-02-07T13:10:27.049086+0000","last_peered":"2024-02-07T13:10:27.049086+0000","last_clean":"2024-02-07T13:10:27.049086+0000","last_became_active":"2024-02-07T13:10:01.070034+0000","last_became_peered":"2024-02-07T13:10:01.070034+0000","last_unstale":"2024-02-07T13:10:27.049086+0000","last_undegraded":"2024-02-07T13:10:27.049086+0000","last_fullsized":"2024-02-07T13:10:27.049086+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:31:14.193163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047379+0000","last_change":"2024-02-07T13:09:57.051282+0000","last_active":"2024-02-07T13:10:27.047379+0000","last_peered":"2024-02-07T13:10:27.047379+0000","last_clean":"2024-02-07T13:10:27.047379+0000","last_became_active":"2024-02-07T13:09:57.050804+0000","last_became_peered":"2024-02-07T13:09:57.050804+0000","last_unstale":"2024-02-07T13:10:27.047379+0000","last_undegraded":"2024-02-07T13:10:27.047379+0000","last_fullsized":"2024-02-07T13:10:27.047379+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:29:10.880281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"31'4","reported_seq":29,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046909+0000","last_change":"2024-02-07T13:09:59.062482+0000","last_active":"2024-02-07T13:10:27.046909+0000","last_peered":"2024-02-07T13:10:27.046909+0000","last_clean":"2024-02-07T13:10:27.046909+0000","last_became_active":"2024-02-07T13:09:59.062176+0000","last_became_peered":"2024-02-07T13:09:59.062176+0000","last_unstale":"2024-02-07T13:10:27.046909+0000","last_undegraded":"2024-02-07T13:10:27.046909+0000","last_fullsized":"2024-02-07T13:10:27.046909+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:49:54.976113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048971+0000","last_change":"2024-02-07T13:10:03.083917+0000","last_active":"2024-02-07T13:10:27.048971+0000","last_peered":"2024-02-07T13:10:27.048971+0000","last_clean":"2024-02-07T13:10:27.048971+0000","last_became_active":"2024-02-07T13:10:03.083203+0000","last_became_peered":"2024-02-07T13:10:03.083203+0000","last_unstale":"2024-02-07T13:10:27.048971+0000","last_undegraded":"2024-02-07T13:10:27.048971+0000","last_fullsized":"2024-02-07T13:10:27.048971+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:51:21.941839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049236+0000","last_change":"2024-02-07T13:10:01.071481+0000","last_active":"2024-02-07T13:10:27.049236+0000","last_peered":"2024-02-07T13:10:27.049236+0000","last_clean":"2024-02-07T13:10:27.049236+0000","last_became_active":"2024-02-07T13:10:01.071067+0000","last_became_peered":"2024-02-07T13:10:01.071067+0000","last_unstale":"2024-02-07T13:10:27.049236+0000","last_undegraded":"2024-02-07T13:10:27.049236+0000","last_fullsized":"2024-02-07T13:10:27.049236+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:42:42.837022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047321+0000","last_change":"2024-02-07T13:09:57.050043+0000","last_active":"2024-02-07T13:10:27.047321+0000","last_peered":"2024-02-07T13:10:27.047321+0000","last_clean":"2024-02-07T13:10:27.047321+0000","last_became_active":"2024-02-07T13:09:57.049253+0000","last_became_peered":"2024-02-07T13:09:57.049253+0000","last_unstale":"2024-02-07T13:10:27.047321+0000","last_undegraded":"2024-02-07T13:10:27.047321+0000","last_fullsized":"2024-02-07T13:10:27.047321+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:19:44.671027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"31'11","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048230+0000","last_change":"2024-02-07T13:09:59.058588+0000","last_active":"2024-02-07T13:10:27.048230+0000","last_peered":"2024-02-07T13:10:27.048230+0000","last_clean":"2024-02-07T13:10:27.048230+0000","last_became_active":"2024-02-07T13:09:59.058179+0000","last_became_peered":"2024-02-07T13:09:59.058179+0000","last_unstale":"2024-02-07T13:10:27.048230+0000","last_undegraded":"2024-02-07T13:10:27.048230+0000","last_fullsized":"2024-02-07T13:10:27.048230+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:13:25.374623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048344+0000","last_change":"2024-02-07T13:10:03.087474+0000","last_active":"2024-02-07T13:10:27.048344+0000","last_peered":"2024-02-07T13:10:27.048344+0000","last_clean":"2024-02-07T13:10:27.048344+0000","last_became_active":"2024-02-07T13:10:03.086938+0000","last_became_peered":"2024-02-07T13:10:03.086938+0000","last_unstale":"2024-02-07T13:10:27.048344+0000","last_undegraded":"2024-02-07T13:10:27.048344+0000","last_fullsized":"2024-02-07T13:10:27.048344+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:21:39.226846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"24'2","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049480+0000","last_change":"2024-02-07T13:09:57.048265+0000","last_active":"2024-02-07T13:10:27.049480+0000","last_peered":"2024-02-07T13:10:27.049480+0000","last_clean":"2024-02-07T13:10:27.049480+0000","last_became_active":"2024-02-07T13:09:57.046991+0000","last_became_peered":"2024-02-07T13:09:57.046991+0000","last_unstale":"2024-02-07T13:10:27.049480+0000","last_undegraded":"2024-02-07T13:10:27.049480+0000","last_fullsized":"2024-02-07T13:10:27.049480+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:07:59.205642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"31'11","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049628+0000","last_change":"2024-02-07T13:09:59.062964+0000","last_active":"2024-02-07T13:10:27.049628+0000","last_peered":"2024-02-07T13:10:27.049628+0000","last_clean":"2024-02-07T13:10:27.049628+0000","last_became_active":"2024-02-07T13:09:59.062609+0000","last_became_peered":"2024-02-07T13:09:59.062609+0000","last_unstale":"2024-02-07T13:10:27.049628+0000","last_undegraded":"2024-02-07T13:10:27.049628+0000","last_fullsized":"2024-02-07T13:10:27.049628+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:01:56.347117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"31'8","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099749+0000","last_change":"2024-02-07T13:10:01.067654+0000","last_active":"2024-02-07T13:10:47.099749+0000","last_peered":"2024-02-07T13:10:47.099749+0000","last_clean":"2024-02-07T13:10:47.099749+0000","last_became_active":"2024-02-07T13:10:01.067289+0000","last_became_peered":"2024-02-07T13:10:01.067289+0000","last_unstale":"2024-02-07T13:10:47.099749+0000","last_undegraded":"2024-02-07T13:10:47.099749+0000","last_fullsized":"2024-02-07T13:10:47.099749+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:37:37.914427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046551+0000","last_change":"2024-02-07T13:10:03.084064+0000","last_active":"2024-02-07T13:10:27.046551+0000","last_peered":"2024-02-07T13:10:27.046551+0000","last_clean":"2024-02-07T13:10:27.046551+0000","last_became_active":"2024-02-07T13:10:03.083677+0000","last_became_peered":"2024-02-07T13:10:03.083677+0000","last_unstale":"2024-02-07T13:10:27.046551+0000","last_undegraded":"2024-02-07T13:10:27.046551+0000","last_fullsized":"2024-02-07T13:10:27.046551+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:04:48.581898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048216+0000","last_change":"2024-02-07T13:09:57.046631+0000","last_active":"2024-02-07T13:10:27.048216+0000","last_peered":"2024-02-07T13:10:27.048216+0000","last_clean":"2024-02-07T13:10:27.048216+0000","last_became_active":"2024-02-07T13:09:57.046183+0000","last_became_peered":"2024-02-07T13:09:57.046183+0000","last_unstale":"2024-02-07T13:10:27.048216+0000","last_undegraded":"2024-02-07T13:10:27.048216+0000","last_fullsized":"2024-02-07T13:10:27.048216+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:08:59.572003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"31'16","reported_seq":57,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099577+0000","last_change":"2024-02-07T13:09:59.058819+0000","last_active":"2024-02-07T13:10:47.099577+0000","last_peered":"2024-02-07T13:10:47.099577+0000","last_clean":"2024-02-07T13:10:47.099577+0000","last_became_active":"2024-02-07T13:09:59.058424+0000","last_became_peered":"2024-02-07T13:09:59.058424+0000","last_unstale":"2024-02-07T13:10:47.099577+0000","last_undegraded":"2024-02-07T13:10:47.099577+0000","last_fullsized":"2024-02-07T13:10:47.099577+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:47:51.736035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047466+0000","last_change":"2024-02-07T13:10:01.069928+0000","last_active":"2024-02-07T13:10:27.047466+0000","last_peered":"2024-02-07T13:10:27.047466+0000","last_clean":"2024-02-07T13:10:27.047466+0000","last_became_active":"2024-02-07T13:10:01.069645+0000","last_became_peered":"2024-02-07T13:10:01.069645+0000","last_unstale":"2024-02-07T13:10:27.047466+0000","last_undegraded":"2024-02-07T13:10:27.047466+0000","last_fullsized":"2024-02-07T13:10:27.047466+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T21:15:12.369561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048151+0000","last_change":"2024-02-07T13:10:03.084303+0000","last_active":"2024-02-07T13:10:27.048151+0000","last_peered":"2024-02-07T13:10:27.048151+0000","last_clean":"2024-02-07T13:10:27.048151+0000","last_became_active":"2024-02-07T13:10:03.083952+0000","last_became_peered":"2024-02-07T13:10:03.083952+0000","last_unstale":"2024-02-07T13:10:27.048151+0000","last_undegraded":"2024-02-07T13:10:27.048151+0000","last_fullsized":"2024-02-07T13:10:27.048151+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:36:58.824048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"24'1","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047737+0000","last_change":"2024-02-07T13:09:57.051899+0000","last_active":"2024-02-07T13:10:27.047737+0000","last_peered":"2024-02-07T13:10:27.047737+0000","last_clean":"2024-02-07T13:10:27.047737+0000","last_became_active":"2024-02-07T13:09:57.051533+0000","last_became_peered":"2024-02-07T13:09:57.051533+0000","last_unstale":"2024-02-07T13:10:27.047737+0000","last_undegraded":"2024-02-07T13:10:27.047737+0000","last_fullsized":"2024-02-07T13:10:27.047737+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:47:57.739666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"31'19","reported_seq":59,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048508+0000","last_change":"2024-02-07T13:09:59.056897+0000","last_active":"2024-02-07T13:10:27.048508+0000","last_peered":"2024-02-07T13:10:27.048508+0000","last_clean":"2024-02-07T13:10:27.048508+0000","last_became_active":"2024-02-07T13:09:59.056509+0000","last_became_peered":"2024-02-07T13:09:59.056509+0000","last_unstale":"2024-02-07T13:10:27.048508+0000","last_undegraded":"2024-02-07T13:10:27.048508+0000","last_fullsized":"2024-02-07T13:10:27.048508+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:35:46.936987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047648+0000","last_change":"2024-02-07T13:10:01.069820+0000","last_active":"2024-02-07T13:10:27.047648+0000","last_peered":"2024-02-07T13:10:27.047648+0000","last_clean":"2024-02-07T13:10:27.047648+0000","last_became_active":"2024-02-07T13:10:01.069514+0000","last_became_peered":"2024-02-07T13:10:01.069514+0000","last_unstale":"2024-02-07T13:10:27.047648+0000","last_undegraded":"2024-02-07T13:10:27.047648+0000","last_fullsized":"2024-02-07T13:10:27.047648+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:56:46.538421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049679+0000","last_change":"2024-02-07T13:10:03.089611+0000","last_active":"2024-02-07T13:10:27.049679+0000","last_peered":"2024-02-07T13:10:27.049679+0000","last_clean":"2024-02-07T13:10:27.049679+0000","last_became_active":"2024-02-07T13:10:03.089206+0000","last_became_peered":"2024-02-07T13:10:03.089206+0000","last_unstale":"2024-02-07T13:10:27.049679+0000","last_undegraded":"2024-02-07T13:10:27.049679+0000","last_fullsized":"2024-02-07T13:10:27.049679+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:56:50.474614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"31'18","reported_seq":55,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047018+0000","last_change":"2024-02-07T13:09:59.063025+0000","last_active":"2024-02-07T13:10:27.047018+0000","last_peered":"2024-02-07T13:10:27.047018+0000","last_clean":"2024-02-07T13:10:27.047018+0000","last_became_active":"2024-02-07T13:09:59.062476+0000","last_became_peered":"2024-02-07T13:09:59.062476+0000","last_unstale":"2024-02-07T13:10:27.047018+0000","last_undegraded":"2024-02-07T13:10:27.047018+0000","last_fullsized":"2024-02-07T13:10:27.047018+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:49:30.957915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046017+0000","last_change":"2024-02-07T13:09:57.044747+0000","last_active":"2024-02-07T13:10:27.046017+0000","last_peered":"2024-02-07T13:10:27.046017+0000","last_clean":"2024-02-07T13:10:27.046017+0000","last_became_active":"2024-02-07T13:09:57.044366+0000","last_became_peered":"2024-02-07T13:09:57.044366+0000","last_unstale":"2024-02-07T13:10:27.046017+0000","last_undegraded":"2024-02-07T13:10:27.046017+0000","last_fullsized":"2024-02-07T13:10:27.046017+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:45:04.878046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049216+0000","last_change":"2024-02-07T13:10:01.074657+0000","last_active":"2024-02-07T13:10:27.049216+0000","last_peered":"2024-02-07T13:10:27.049216+0000","last_clean":"2024-02-07T13:10:27.049216+0000","last_became_active":"2024-02-07T13:10:01.074241+0000","last_became_peered":"2024-02-07T13:10:01.074241+0000","last_unstale":"2024-02-07T13:10:27.049216+0000","last_undegraded":"2024-02-07T13:10:27.049216+0000","last_fullsized":"2024-02-07T13:10:27.049216+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:55:29.042308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047581+0000","last_change":"2024-02-07T13:10:03.087241+0000","last_active":"2024-02-07T13:10:27.047581+0000","last_peered":"2024-02-07T13:10:27.047581+0000","last_clean":"2024-02-07T13:10:27.047581+0000","last_became_active":"2024-02-07T13:10:03.086928+0000","last_became_peered":"2024-02-07T13:10:03.086928+0000","last_unstale":"2024-02-07T13:10:27.047581+0000","last_undegraded":"2024-02-07T13:10:27.047581+0000","last_fullsized":"2024-02-07T13:10:27.047581+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:39:10.461534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"31'10","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046466+0000","last_change":"2024-02-07T13:09:59.058663+0000","last_active":"2024-02-07T13:10:27.046466+0000","last_peered":"2024-02-07T13:10:27.046466+0000","last_clean":"2024-02-07T13:10:27.046466+0000","last_became_active":"2024-02-07T13:09:59.058350+0000","last_became_peered":"2024-02-07T13:09:59.058350+0000","last_unstale":"2024-02-07T13:10:27.046466+0000","last_undegraded":"2024-02-07T13:10:27.046466+0000","last_fullsized":"2024-02-07T13:10:27.046466+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T01:05:26.891836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048268+0000","last_change":"2024-02-07T13:09:57.052708+0000","last_active":"2024-02-07T13:10:27.048268+0000","last_peered":"2024-02-07T13:10:27.048268+0000","last_clean":"2024-02-07T13:10:27.048268+0000","last_became_active":"2024-02-07T13:09:57.052338+0000","last_became_peered":"2024-02-07T13:09:57.052338+0000","last_unstale":"2024-02-07T13:10:27.048268+0000","last_undegraded":"2024-02-07T13:10:27.048268+0000","last_fullsized":"2024-02-07T13:10:27.048268+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:54:49.812290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"32'208","reported_seq":314,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:29.160902+0000","last_change":"2024-02-07T13:09:47.188747+0000","last_active":"2024-02-07T13:10:29.160902+0000","last_peered":"2024-02-07T13:10:29.160902+0000","last_clean":"2024-02-07T13:10:29.160902+0000","last_became_active":"2024-02-07T13:09:47.188364+0000","last_became_peered":"2024-02-07T13:09:47.188364+0000","last_unstale":"2024-02-07T13:10:29.160902+0000","last_undegraded":"2024-02-07T13:10:29.160902+0000","last_fullsized":"2024-02-07T13:10:29.160902+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:46.169165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:46.169165+0000","last_clean_scrub_stamp":"2024-02-07T13:09:46.169165+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:42:48.929160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048825+0000","last_change":"2024-02-07T13:10:01.069708+0000","last_active":"2024-02-07T13:10:27.048825+0000","last_peered":"2024-02-07T13:10:27.048825+0000","last_clean":"2024-02-07T13:10:27.048825+0000","last_became_active":"2024-02-07T13:10:01.069389+0000","last_became_peered":"2024-02-07T13:10:01.069389+0000","last_unstale":"2024-02-07T13:10:27.048825+0000","last_undegraded":"2024-02-07T13:10:27.048825+0000","last_fullsized":"2024-02-07T13:10:27.048825+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:00:15.026029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048897+0000","last_change":"2024-02-07T13:10:03.086629+0000","last_active":"2024-02-07T13:10:27.048897+0000","last_peered":"2024-02-07T13:10:27.048897+0000","last_clean":"2024-02-07T13:10:27.048897+0000","last_became_active":"2024-02-07T13:10:03.085398+0000","last_became_peered":"2024-02-07T13:10:03.085398+0000","last_unstale":"2024-02-07T13:10:27.048897+0000","last_undegraded":"2024-02-07T13:10:27.048897+0000","last_fullsized":"2024-02-07T13:10:27.048897+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:52:40.751196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"31'14","reported_seq":44,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046250+0000","last_change":"2024-02-07T13:09:59.059584+0000","last_active":"2024-02-07T13:10:27.046250+0000","last_peered":"2024-02-07T13:10:27.046250+0000","last_clean":"2024-02-07T13:10:27.046250+0000","last_became_active":"2024-02-07T13:09:59.059247+0000","last_became_peered":"2024-02-07T13:09:59.059247+0000","last_unstale":"2024-02-07T13:10:27.046250+0000","last_undegraded":"2024-02-07T13:10:27.046250+0000","last_fullsized":"2024-02-07T13:10:27.046250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:39:48.582929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046997+0000","last_change":"2024-02-07T13:09:57.053260+0000","last_active":"2024-02-07T13:10:27.046997+0000","last_peered":"2024-02-07T13:10:27.046997+0000","last_clean":"2024-02-07T13:10:27.046997+0000","last_became_active":"2024-02-07T13:09:57.052900+0000","last_became_peered":"2024-02-07T13:09:57.052900+0000","last_unstale":"2024-02-07T13:10:27.046997+0000","last_undegraded":"2024-02-07T13:10:27.046997+0000","last_fullsized":"2024-02-07T13:10:27.046997+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:16:28.463232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048202+0000","last_change":"2024-02-07T13:10:01.067894+0000","last_active":"2024-02-07T13:10:27.048202+0000","last_peered":"2024-02-07T13:10:27.048202+0000","last_clean":"2024-02-07T13:10:27.048202+0000","last_became_active":"2024-02-07T13:10:01.067241+0000","last_became_peered":"2024-02-07T13:10:01.067241+0000","last_unstale":"2024-02-07T13:10:27.048202+0000","last_undegraded":"2024-02-07T13:10:27.048202+0000","last_fullsized":"2024-02-07T13:10:27.048202+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:15:30.741385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047641+0000","last_change":"2024-02-07T13:10:03.083440+0000","last_active":"2024-02-07T13:10:27.047641+0000","last_peered":"2024-02-07T13:10:27.047641+0000","last_clean":"2024-02-07T13:10:27.047641+0000","last_became_active":"2024-02-07T13:10:03.082905+0000","last_became_peered":"2024-02-07T13:10:03.082905+0000","last_unstale":"2024-02-07T13:10:27.047641+0000","last_undegraded":"2024-02-07T13:10:27.047641+0000","last_fullsized":"2024-02-07T13:10:27.047641+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:12:50.039813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049448+0000","last_change":"2024-02-07T13:09:57.050153+0000","last_active":"2024-02-07T13:10:27.049448+0000","last_peered":"2024-02-07T13:10:27.049448+0000","last_clean":"2024-02-07T13:10:27.049448+0000","last_became_active":"2024-02-07T13:09:57.049747+0000","last_became_peered":"2024-02-07T13:09:57.049747+0000","last_unstale":"2024-02-07T13:10:27.049448+0000","last_undegraded":"2024-02-07T13:10:27.049448+0000","last_fullsized":"2024-02-07T13:10:27.049448+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:45:59.518112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"31'28","reported_seq":83,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099540+0000","last_change":"2024-02-07T13:09:59.061206+0000","last_active":"2024-02-07T13:10:47.099540+0000","last_peered":"2024-02-07T13:10:47.099540+0000","last_clean":"2024-02-07T13:10:47.099540+0000","last_became_active":"2024-02-07T13:09:59.060778+0000","last_became_peered":"2024-02-07T13:09:59.060778+0000","last_unstale":"2024-02-07T13:10:47.099540+0000","last_undegraded":"2024-02-07T13:10:47.099540+0000","last_fullsized":"2024-02-07T13:10:47.099540+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:10:00.044490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"31'8","reported_seq":42,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099735+0000","last_change":"2024-02-07T13:10:01.069065+0000","last_active":"2024-02-07T13:10:47.099735+0000","last_peered":"2024-02-07T13:10:47.099735+0000","last_clean":"2024-02-07T13:10:47.099735+0000","last_became_active":"2024-02-07T13:10:01.068194+0000","last_became_peered":"2024-02-07T13:10:01.068194+0000","last_unstale":"2024-02-07T13:10:47.099735+0000","last_undegraded":"2024-02-07T13:10:47.099735+0000","last_fullsized":"2024-02-07T13:10:47.099735+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:35:57.819339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047054+0000","last_change":"2024-02-07T13:10:03.084644+0000","last_active":"2024-02-07T13:10:27.047054+0000","last_peered":"2024-02-07T13:10:27.047054+0000","last_clean":"2024-02-07T13:10:27.047054+0000","last_became_active":"2024-02-07T13:10:03.084040+0000","last_became_peered":"2024-02-07T13:10:03.084040+0000","last_unstale":"2024-02-07T13:10:27.047054+0000","last_undegraded":"2024-02-07T13:10:27.047054+0000","last_fullsized":"2024-02-07T13:10:27.047054+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:36:04.508794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"24'1","reported_seq":28,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049813+0000","last_change":"2024-02-07T13:09:57.046346+0000","last_active":"2024-02-07T13:10:27.049813+0000","last_peered":"2024-02-07T13:10:27.049813+0000","last_clean":"2024-02-07T13:10:27.049813+0000","last_became_active":"2024-02-07T13:09:57.045714+0000","last_became_peered":"2024-02-07T13:09:57.045714+0000","last_unstale":"2024-02-07T13:10:27.049813+0000","last_undegraded":"2024-02-07T13:10:27.049813+0000","last_fullsized":"2024-02-07T13:10:27.049813+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:30:58.986680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"31'13","reported_seq":50,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049679+0000","last_change":"2024-02-07T13:09:59.062189+0000","last_active":"2024-02-07T13:10:27.049679+0000","last_peered":"2024-02-07T13:10:27.049679+0000","last_clean":"2024-02-07T13:10:27.049679+0000","last_became_active":"2024-02-07T13:09:59.061919+0000","last_became_peered":"2024-02-07T13:09:59.061919+0000","last_unstale":"2024-02-07T13:10:27.049679+0000","last_undegraded":"2024-02-07T13:10:27.049679+0000","last_fullsized":"2024-02-07T13:10:27.049679+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:35:27.096214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"31'8","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099514+0000","last_change":"2024-02-07T13:10:01.074761+0000","last_active":"2024-02-07T13:10:47.099514+0000","last_peered":"2024-02-07T13:10:47.099514+0000","last_clean":"2024-02-07T13:10:47.099514+0000","last_became_active":"2024-02-07T13:10:01.074293+0000","last_became_peered":"2024-02-07T13:10:01.074293+0000","last_unstale":"2024-02-07T13:10:47.099514+0000","last_undegraded":"2024-02-07T13:10:47.099514+0000","last_fullsized":"2024-02-07T13:10:47.099514+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:02:14.719921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047868+0000","last_change":"2024-02-07T13:10:03.085019+0000","last_active":"2024-02-07T13:10:27.047868+0000","last_peered":"2024-02-07T13:10:27.047868+0000","last_clean":"2024-02-07T13:10:27.047868+0000","last_became_active":"2024-02-07T13:10:03.084531+0000","last_became_peered":"2024-02-07T13:10:03.084531+0000","last_unstale":"2024-02-07T13:10:27.047868+0000","last_undegraded":"2024-02-07T13:10:27.047868+0000","last_fullsized":"2024-02-07T13:10:27.047868+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:40:51.067114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049284+0000","last_change":"2024-02-07T13:09:57.052011+0000","last_active":"2024-02-07T13:10:27.049284+0000","last_peered":"2024-02-07T13:10:27.049284+0000","last_clean":"2024-02-07T13:10:27.049284+0000","last_became_active":"2024-02-07T13:09:57.051567+0000","last_became_peered":"2024-02-07T13:09:57.051567+0000","last_unstale":"2024-02-07T13:10:27.049284+0000","last_undegraded":"2024-02-07T13:10:27.049284+0000","last_fullsized":"2024-02-07T13:10:27.049284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:55:55.217672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"31'12","reported_seq":41,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049345+0000","last_change":"2024-02-07T13:09:59.061119+0000","last_active":"2024-02-07T13:10:27.049345+0000","last_peered":"2024-02-07T13:10:27.049345+0000","last_clean":"2024-02-07T13:10:27.049345+0000","last_became_active":"2024-02-07T13:09:59.060310+0000","last_became_peered":"2024-02-07T13:09:59.060310+0000","last_unstale":"2024-02-07T13:10:27.049345+0000","last_undegraded":"2024-02-07T13:10:27.049345+0000","last_fullsized":"2024-02-07T13:10:27.049345+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:57:03.247934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047370+0000","last_change":"2024-02-07T13:10:01.069945+0000","last_active":"2024-02-07T13:10:27.047370+0000","last_peered":"2024-02-07T13:10:27.047370+0000","last_clean":"2024-02-07T13:10:27.047370+0000","last_became_active":"2024-02-07T13:10:01.069571+0000","last_became_peered":"2024-02-07T13:10:01.069571+0000","last_unstale":"2024-02-07T13:10:27.047370+0000","last_undegraded":"2024-02-07T13:10:27.047370+0000","last_fullsized":"2024-02-07T13:10:27.047370+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:24:02.546134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048741+0000","last_change":"2024-02-07T13:10:03.084016+0000","last_active":"2024-02-07T13:10:27.048741+0000","last_peered":"2024-02-07T13:10:27.048741+0000","last_clean":"2024-02-07T13:10:27.048741+0000","last_became_active":"2024-02-07T13:10:03.083682+0000","last_became_peered":"2024-02-07T13:10:03.083682+0000","last_unstale":"2024-02-07T13:10:27.048741+0000","last_undegraded":"2024-02-07T13:10:27.048741+0000","last_fullsized":"2024-02-07T13:10:27.048741+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:27:54.916855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"31'8","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099532+0000","last_change":"2024-02-07T13:10:01.070244+0000","last_active":"2024-02-07T13:10:47.099532+0000","last_peered":"2024-02-07T13:10:47.099532+0000","last_clean":"2024-02-07T13:10:47.099532+0000","last_became_active":"2024-02-07T13:10:01.069936+0000","last_became_peered":"2024-02-07T13:10:01.069936+0000","last_unstale":"2024-02-07T13:10:47.099532+0000","last_undegraded":"2024-02-07T13:10:47.099532+0000","last_fullsized":"2024-02-07T13:10:47.099532+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:01:47.717905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047095+0000","last_change":"2024-02-07T13:09:57.053104+0000","last_active":"2024-02-07T13:10:27.047095+0000","last_peered":"2024-02-07T13:10:27.047095+0000","last_clean":"2024-02-07T13:10:27.047095+0000","last_became_active":"2024-02-07T13:09:57.052636+0000","last_became_peered":"2024-02-07T13:09:57.052636+0000","last_unstale":"2024-02-07T13:10:27.047095+0000","last_undegraded":"2024-02-07T13:10:27.047095+0000","last_fullsized":"2024-02-07T13:10:27.047095+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:39:32.298984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"31'12","reported_seq":46,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048563+0000","last_change":"2024-02-07T13:09:59.058803+0000","last_active":"2024-02-07T13:10:27.048563+0000","last_peered":"2024-02-07T13:10:27.048563+0000","last_clean":"2024-02-07T13:10:27.048563+0000","last_became_active":"2024-02-07T13:09:59.058507+0000","last_became_peered":"2024-02-07T13:09:59.058507+0000","last_unstale":"2024-02-07T13:10:27.048563+0000","last_undegraded":"2024-02-07T13:10:27.048563+0000","last_fullsized":"2024-02-07T13:10:27.048563+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:32:51.490644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049282+0000","last_change":"2024-02-07T13:10:03.087204+0000","last_active":"2024-02-07T13:10:27.049282+0000","last_peered":"2024-02-07T13:10:27.049282+0000","last_clean":"2024-02-07T13:10:27.049282+0000","last_became_active":"2024-02-07T13:10:03.086868+0000","last_became_peered":"2024-02-07T13:10:03.086868+0000","last_unstale":"2024-02-07T13:10:27.049282+0000","last_undegraded":"2024-02-07T13:10:27.049282+0000","last_fullsized":"2024-02-07T13:10:27.049282+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:23:31.655829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049127+0000","last_change":"2024-02-07T13:10:01.073468+0000","last_active":"2024-02-07T13:10:27.049127+0000","last_peered":"2024-02-07T13:10:27.049127+0000","last_clean":"2024-02-07T13:10:27.049127+0000","last_became_active":"2024-02-07T13:10:01.073188+0000","last_became_peered":"2024-02-07T13:10:01.073188+0000","last_unstale":"2024-02-07T13:10:27.049127+0000","last_undegraded":"2024-02-07T13:10:27.049127+0000","last_fullsized":"2024-02-07T13:10:27.049127+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:29:44.593605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046127+0000","last_change":"2024-02-07T13:09:57.044721+0000","last_active":"2024-02-07T13:10:27.046127+0000","last_peered":"2024-02-07T13:10:27.046127+0000","last_clean":"2024-02-07T13:10:27.046127+0000","last_became_active":"2024-02-07T13:09:57.044373+0000","last_became_peered":"2024-02-07T13:09:57.044373+0000","last_unstale":"2024-02-07T13:10:27.046127+0000","last_undegraded":"2024-02-07T13:10:27.046127+0000","last_fullsized":"2024-02-07T13:10:27.046127+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T22:30:25.307104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"31'15","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047327+0000","last_change":"2024-02-07T13:09:59.057454+0000","last_active":"2024-02-07T13:10:27.047327+0000","last_peered":"2024-02-07T13:10:27.047327+0000","last_clean":"2024-02-07T13:10:27.047327+0000","last_became_active":"2024-02-07T13:09:59.057125+0000","last_became_peered":"2024-02-07T13:09:59.057125+0000","last_unstale":"2024-02-07T13:10:27.047327+0000","last_undegraded":"2024-02-07T13:10:27.047327+0000","last_fullsized":"2024-02-07T13:10:27.047327+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:22:17.016717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048520+0000","last_change":"2024-02-07T13:10:03.087638+0000","last_active":"2024-02-07T13:10:27.048520+0000","last_peered":"2024-02-07T13:10:27.048520+0000","last_clean":"2024-02-07T13:10:27.048520+0000","last_became_active":"2024-02-07T13:10:03.086919+0000","last_became_peered":"2024-02-07T13:10:03.086919+0000","last_unstale":"2024-02-07T13:10:27.048520+0000","last_undegraded":"2024-02-07T13:10:27.048520+0000","last_fullsized":"2024-02-07T13:10:27.048520+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T14:02:54.902460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047977+0000","last_change":"2024-02-07T13:09:57.047170+0000","last_active":"2024-02-07T13:10:27.047977+0000","last_peered":"2024-02-07T13:10:27.047977+0000","last_clean":"2024-02-07T13:10:27.047977+0000","last_became_active":"2024-02-07T13:09:57.046766+0000","last_became_peered":"2024-02-07T13:09:57.046766+0000","last_unstale":"2024-02-07T13:10:27.047977+0000","last_undegraded":"2024-02-07T13:10:27.047977+0000","last_fullsized":"2024-02-07T13:10:27.047977+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:36:36.062045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"31'9","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046904+0000","last_change":"2024-02-07T13:09:59.059949+0000","last_active":"2024-02-07T13:10:27.046904+0000","last_peered":"2024-02-07T13:10:27.046904+0000","last_clean":"2024-02-07T13:10:27.046904+0000","last_became_active":"2024-02-07T13:09:59.059603+0000","last_became_peered":"2024-02-07T13:09:59.059603+0000","last_unstale":"2024-02-07T13:10:27.046904+0000","last_undegraded":"2024-02-07T13:10:27.046904+0000","last_fullsized":"2024-02-07T13:10:27.046904+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:32:26.800915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048746+0000","last_change":"2024-02-07T13:10:01.069673+0000","last_active":"2024-02-07T13:10:27.048746+0000","last_peered":"2024-02-07T13:10:27.048746+0000","last_clean":"2024-02-07T13:10:27.048746+0000","last_became_active":"2024-02-07T13:10:01.069289+0000","last_became_peered":"2024-02-07T13:10:01.069289+0000","last_unstale":"2024-02-07T13:10:27.048746+0000","last_undegraded":"2024-02-07T13:10:27.048746+0000","last_fullsized":"2024-02-07T13:10:27.048746+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T00:14:44.669314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047292+0000","last_change":"2024-02-07T13:10:03.084007+0000","last_active":"2024-02-07T13:10:27.047292+0000","last_peered":"2024-02-07T13:10:27.047292+0000","last_clean":"2024-02-07T13:10:27.047292+0000","last_became_active":"2024-02-07T13:10:03.083199+0000","last_became_peered":"2024-02-07T13:10:03.083199+0000","last_unstale":"2024-02-07T13:10:27.047292+0000","last_undegraded":"2024-02-07T13:10:27.047292+0000","last_fullsized":"2024-02-07T13:10:27.047292+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-09T01:04:59.002423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"31'8","reported_seq":39,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:47.099489+0000","last_change":"2024-02-07T13:10:01.071049+0000","last_active":"2024-02-07T13:10:47.099489+0000","last_peered":"2024-02-07T13:10:47.099489+0000","last_clean":"2024-02-07T13:10:47.099489+0000","last_became_active":"2024-02-07T13:10:01.070635+0000","last_became_peered":"2024-02-07T13:10:01.070635+0000","last_unstale":"2024-02-07T13:10:47.099489+0000","last_undegraded":"2024-02-07T13:10:47.099489+0000","last_fullsized":"2024-02-07T13:10:47.099489+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:09:34.130416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048536+0000","last_change":"2024-02-07T13:09:57.048948+0000","last_active":"2024-02-07T13:10:27.048536+0000","last_peered":"2024-02-07T13:10:27.048536+0000","last_clean":"2024-02-07T13:10:27.048536+0000","last_became_active":"2024-02-07T13:09:57.047957+0000","last_became_peered":"2024-02-07T13:09:57.047957+0000","last_unstale":"2024-02-07T13:10:27.048536+0000","last_undegraded":"2024-02-07T13:10:27.048536+0000","last_fullsized":"2024-02-07T13:10:27.048536+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T20:32:57.617385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"31'19","reported_seq":54,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047911+0000","last_change":"2024-02-07T13:09:59.057023+0000","last_active":"2024-02-07T13:10:27.047911+0000","last_peered":"2024-02-07T13:10:27.047911+0000","last_clean":"2024-02-07T13:10:27.047911+0000","last_became_active":"2024-02-07T13:09:59.056628+0000","last_became_peered":"2024-02-07T13:09:59.056628+0000","last_unstale":"2024-02-07T13:10:27.047911+0000","last_undegraded":"2024-02-07T13:10:27.047911+0000","last_fullsized":"2024-02-07T13:10:27.047911+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T16:57:05.923715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047617+0000","last_change":"2024-02-07T13:10:03.084832+0000","last_active":"2024-02-07T13:10:27.047617+0000","last_peered":"2024-02-07T13:10:27.047617+0000","last_clean":"2024-02-07T13:10:27.047617+0000","last_became_active":"2024-02-07T13:10:03.084327+0000","last_became_peered":"2024-02-07T13:10:03.084327+0000","last_unstale":"2024-02-07T13:10:27.047617+0000","last_undegraded":"2024-02-07T13:10:27.047617+0000","last_fullsized":"2024-02-07T13:10:27.047617+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:40:14.035731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048642+0000","last_change":"2024-02-07T13:09:57.048052+0000","last_active":"2024-02-07T13:10:27.048642+0000","last_peered":"2024-02-07T13:10:27.048642+0000","last_clean":"2024-02-07T13:10:27.048642+0000","last_became_active":"2024-02-07T13:09:57.046940+0000","last_became_peered":"2024-02-07T13:09:57.046940+0000","last_unstale":"2024-02-07T13:10:27.048642+0000","last_undegraded":"2024-02-07T13:10:27.048642+0000","last_fullsized":"2024-02-07T13:10:27.048642+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T15:44:53.465201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"31'17","reported_seq":51,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049251+0000","last_change":"2024-02-07T13:09:59.062479+0000","last_active":"2024-02-07T13:10:27.049251+0000","last_peered":"2024-02-07T13:10:27.049251+0000","last_clean":"2024-02-07T13:10:27.049251+0000","last_became_active":"2024-02-07T13:09:59.062131+0000","last_became_peered":"2024-02-07T13:09:59.062131+0000","last_unstale":"2024-02-07T13:10:27.049251+0000","last_undegraded":"2024-02-07T13:10:27.049251+0000","last_fullsized":"2024-02-07T13:10:27.049251+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T17:04:07.666771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047482+0000","last_change":"2024-02-07T13:10:01.071535+0000","last_active":"2024-02-07T13:10:27.047482+0000","last_peered":"2024-02-07T13:10:27.047482+0000","last_clean":"2024-02-07T13:10:27.047482+0000","last_became_active":"2024-02-07T13:10:01.071205+0000","last_became_peered":"2024-02-07T13:10:01.071205+0000","last_unstale":"2024-02-07T13:10:27.047482+0000","last_undegraded":"2024-02-07T13:10:27.047482+0000","last_fullsized":"2024-02-07T13:10:27.047482+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:06:34.933639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.045967+0000","last_change":"2024-02-07T13:10:03.082900+0000","last_active":"2024-02-07T13:10:27.045967+0000","last_peered":"2024-02-07T13:10:27.045967+0000","last_clean":"2024-02-07T13:10:27.045967+0000","last_became_active":"2024-02-07T13:10:03.082386+0000","last_became_peered":"2024-02-07T13:10:03.082386+0000","last_unstale":"2024-02-07T13:10:27.045967+0000","last_undegraded":"2024-02-07T13:10:27.045967+0000","last_fullsized":"2024-02-07T13:10:27.045967+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:19:53.241008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048768+0000","last_change":"2024-02-07T13:09:57.050107+0000","last_active":"2024-02-07T13:10:27.048768+0000","last_peered":"2024-02-07T13:10:27.048768+0000","last_clean":"2024-02-07T13:10:27.048768+0000","last_became_active":"2024-02-07T13:09:57.049309+0000","last_became_peered":"2024-02-07T13:09:57.049309+0000","last_unstale":"2024-02-07T13:10:27.048768+0000","last_undegraded":"2024-02-07T13:10:27.048768+0000","last_fullsized":"2024-02-07T13:10:27.048768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T19:18:09.378380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"31'10","reported_seq":38,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.048992+0000","last_change":"2024-02-07T13:09:59.059297+0000","last_active":"2024-02-07T13:10:27.048992+0000","last_peered":"2024-02-07T13:10:27.048992+0000","last_clean":"2024-02-07T13:10:27.048992+0000","last_became_active":"2024-02-07T13:09:59.058474+0000","last_became_peered":"2024-02-07T13:09:59.058474+0000","last_unstale":"2024-02-07T13:10:27.048992+0000","last_undegraded":"2024-02-07T13:10:27.048992+0000","last_fullsized":"2024-02-07T13:10:27.048992+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:59:10.990587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046697+0000","last_change":"2024-02-07T13:10:01.069709+0000","last_active":"2024-02-07T13:10:27.046697+0000","last_peered":"2024-02-07T13:10:27.046697+0000","last_clean":"2024-02-07T13:10:27.046697+0000","last_became_active":"2024-02-07T13:10:01.069398+0000","last_became_peered":"2024-02-07T13:10:01.069398+0000","last_unstale":"2024-02-07T13:10:27.046697+0000","last_undegraded":"2024-02-07T13:10:27.046697+0000","last_fullsized":"2024-02-07T13:10:27.046697+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:13:01.382113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.046901+0000","last_change":"2024-02-07T13:10:03.085552+0000","last_active":"2024-02-07T13:10:27.046901+0000","last_peered":"2024-02-07T13:10:27.046901+0000","last_clean":"2024-02-07T13:10:27.046901+0000","last_became_active":"2024-02-07T13:10:03.085225+0000","last_became_peered":"2024-02-07T13:10:03.085225+0000","last_unstale":"2024-02-07T13:10:27.046901+0000","last_undegraded":"2024-02-07T13:10:27.046901+0000","last_fullsized":"2024-02-07T13:10:27.046901+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:02.067405+0000","last_clean_scrub_stamp":"2024-02-07T13:10:02.067405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:55:41.888615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":27,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049477+0000","last_change":"2024-02-07T13:09:57.046664+0000","last_active":"2024-02-07T13:10:27.049477+0000","last_peered":"2024-02-07T13:10:27.049477+0000","last_clean":"2024-02-07T13:10:27.049477+0000","last_became_active":"2024-02-07T13:09:57.045882+0000","last_became_peered":"2024-02-07T13:09:57.045882+0000","last_unstale":"2024-02-07T13:10:27.049477+0000","last_undegraded":"2024-02-07T13:10:27.049477+0000","last_fullsized":"2024-02-07T13:10:27.049477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:56.025363+0000","last_clean_scrub_stamp":"2024-02-07T13:09:56.025363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T13:48:29.700282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"31'15","reported_seq":48,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.049365+0000","last_change":"2024-02-07T13:09:59.059244+0000","last_active":"2024-02-07T13:10:27.049365+0000","last_peered":"2024-02-07T13:10:27.049365+0000","last_clean":"2024-02-07T13:10:27.049365+0000","last_became_active":"2024-02-07T13:09:59.058912+0000","last_became_peered":"2024-02-07T13:09:59.058912+0000","last_unstale":"2024-02-07T13:10:27.049365+0000","last_undegraded":"2024-02-07T13:10:27.049365+0000","last_fullsized":"2024-02-07T13:10:27.049365+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:09:58.035849+0000","last_clean_scrub_stamp":"2024-02-07T13:09:58.035849+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T23:05:23.210608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-02-07T13:10:27.047584+0000","last_change":"2024-02-07T13:10:01.068095+0000","last_active":"2024-02-07T13:10:27.047584+0000","last_peered":"2024-02-07T13:10:27.047584+0000","last_clean":"2024-02-07T13:10:27.047584+0000","last_became_active":"2024-02-07T13:10:01.067769+0000","last_became_peered":"2024-02-07T13:10:01.067769+0000","last_unstale":"2024-02-07T13:10:27.047584+0000","last_undegraded":"2024-02-07T13:10:27.047584+0000","last_fullsized":"2024-02-07T13:10:27.047584+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":27,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-07T13:10:00.054913+0000","last_clean_scrub_stamp":"2024-02-07T13:10:00.054913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-08T18:54:13.238916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411345,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708840,"statfs":{"total":95990841344,"available":95957852160,"internally_reserved":0,"allocated":1060864,"data_stored":703110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 7 13:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"last":0.307}]},{"osd":1,"last update":"Wed Feb 7 13:10:46 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.32700000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738395,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708844,"statfs":{"total":95990841344,"available":95957856256,"internally_reserved":0,"allocated":1060864,"data_stored":703110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Feb 7 13:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.33300000000000002}]},{"osd":2,"last update":"Wed Feb 7 13:10:44 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.49199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708844,"statfs":{"total":95990841344,"available":95957856256,"internally_reserved":0,"allocated":1060864,"data_stored":703110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Feb 7 13:10:16 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.31}]},{"osd":2,"last update":"Wed Feb 7 13:10:47 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.37},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.38300000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-07T13:10:59.149 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-07T13:10:59.149 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-07T13:10:59.149 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-07T13:10:59.150 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- ceph health --format=json 2024-02-07T13:10:59.368 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:58 smithi191 ceph-mon[27623]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:10:59.393 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:10:59.685 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:10:59 smithi159 systemd[1]: Starting Ceph grafana.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:10:59.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:10:59 smithi159 ceph-mon[27461]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:10:59.943 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:10:59 smithi159 podman[38230]: 2024-02-07 13:10:59.685418002 +0000 UTC m=+0.104508729 container create 1eb011bebf11914ce2d9ef7b5274d9872d0d0eb054609ea53bfab0941ae4ea76 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, release=239.1651231664, vendor=Red Hat, Inc., io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=ubi8, description=Ceph Grafana Container, distribution-scope=public, io.buildah.version=1.32.0, architecture=x86_64, version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, build-date=2022-04-29T11:48:11.055897, io.openshift.expose-services=, com.redhat.component=ubi8-container, maintainer=Paul Cuzner , url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-type=git) 2024-02-07T13:10:59.944 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:10:59 smithi159 podman[38230]: 2024-02-07 13:10:59.597032004 +0000 UTC m=+0.016122694 image pull a0ea5e3c46d9434bd173d585b7c2638c7451eaf2543d1d22f9e78442615aaffb quay.io/ceph/ceph-grafana:9.4.12 2024-02-07T13:10:59.944 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:10:59 smithi159 podman[38230]: 2024-02-07 13:10:59.878927776 +0000 UTC m=+0.298018472 container init 1eb011bebf11914ce2d9ef7b5274d9872d0d0eb054609ea53bfab0941ae4ea76 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a, release=239.1651231664, vendor=Red Hat, Inc., io.openshift.expose-services=, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., name=ubi8, build-date=2022-04-29T11:48:11.055897, io.openshift.tags=base rhel8, io.k8s.display-name=Red Hat Universal Base Image 8, version=8.5, com.redhat.component=ubi8-container, maintainer=Paul Cuzner , architecture=x86_64, distribution-scope=public, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-type=git, description=Ceph Grafana Container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.buildah.version=1.32.0) 2024-02-07T13:10:59.944 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:10:59 smithi159 podman[38230]: 2024-02-07 13:10:59.884718592 +0000 UTC m=+0.303809277 container start 1eb011bebf11914ce2d9ef7b5274d9872d0d0eb054609ea53bfab0941ae4ea76 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vendor=Red Hat, Inc., summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, io.openshift.tags=base rhel8, io.k8s.display-name=Red Hat Universal Base Image 8, distribution-scope=public, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, release=239.1651231664, maintainer=Paul Cuzner , version=8.5, name=ubi8, description=Ceph Grafana Container, com.redhat.component=ubi8-container, architecture=x86_64, io.openshift.expose-services=, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.32.0, build-date=2022-04-29T11:48:11.055897) 2024-02-07T13:11:00.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:10:59 smithi067 ceph-mon[25698]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:11:00.238 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:10:59 smithi159 bash[38230]: 1eb011bebf11914ce2d9ef7b5274d9872d0d0eb054609ea53bfab0941ae4ea76 2024-02-07T13:11:00.238 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:10:59 smithi159 systemd[1]: Started Ceph grafana.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:11:00.238 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=settings t=2024-02-07T13:11:00.035909466Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-02-07T13:11:00.238 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=settings t=2024-02-07T13:11:00.036123785Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-02-07T13:11:00.238 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=settings t=2024-02-07T13:11:00.036144863Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-02-07T13:11:00.238 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=settings t=2024-02-07T13:11:00.036152971Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-02-07T13:11:00.238 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=settings t=2024-02-07T13:11:00.036159485Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-07T13:11:00.239 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=settings t=2024-02-07T13:11:00.036166852Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-07T13:11:00.239 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=settings t=2024-02-07T13:11:00.036173901Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-07T13:11:00.239 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=settings t=2024-02-07T13:11:00.0361844Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-07T13:11:00.239 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=settings t=2024-02-07T13:11:00.036190835Z level=info msg="Path Home" path=/usr/share/grafana 2024-02-07T13:11:00.239 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=settings t=2024-02-07T13:11:00.03619701Z level=info msg="Path Data" path=/var/lib/grafana 2024-02-07T13:11:00.239 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=settings t=2024-02-07T13:11:00.036203401Z level=info msg="Path Logs" path=/var/log/grafana 2024-02-07T13:11:00.239 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=settings t=2024-02-07T13:11:00.036209412Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-02-07T13:11:00.239 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=settings t=2024-02-07T13:11:00.03621568Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-02-07T13:11:00.239 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=settings t=2024-02-07T13:11:00.036221979Z level=info msg="App mode production" 2024-02-07T13:11:00.239 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=sqlstore t=2024-02-07T13:11:00.036269152Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-02-07T13:11:00.239 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=sqlstore t=2024-02-07T13:11:00.036286402Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-07T13:11:00.239 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.037174676Z level=info msg="Starting DB migrations" 2024-02-07T13:11:00.239 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.037258989Z level=info msg="Executing migration" id="create migration_log table" 2024-02-07T13:11:00.240 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.037782225Z level=info msg="Executing migration" id="create user table" 2024-02-07T13:11:00.240 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.038234513Z level=info msg="Executing migration" id="add unique index user.login" 2024-02-07T13:11:00.240 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.038729414Z level=info msg="Executing migration" id="add unique index user.email" 2024-02-07T13:11:00.240 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.039228717Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-02-07T13:11:00.240 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.039801879Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-02-07T13:11:00.240 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.040227806Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-02-07T13:11:00.240 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.041000034Z level=info msg="Executing migration" id="create user table v2" 2024-02-07T13:11:00.240 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.041438816Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-02-07T13:11:00.240 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.041869062Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-02-07T13:11:00.240 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.042596215Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-02-07T13:11:00.240 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.042965654Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-02-07T13:11:00.240 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.043328765Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-02-07T13:11:00.240 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.043715075Z level=info msg="Executing migration" id="Update user table charset" 2024-02-07T13:11:00.240 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.043999403Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-02-07T13:11:00.241 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.044408288Z level=info msg="Executing migration" id="Add missing user data" 2024-02-07T13:11:00.241 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.044729398Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-02-07T13:11:00.241 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.04515277Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-02-07T13:11:00.241 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.046080627Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-02-07T13:11:00.241 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.046508447Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-02-07T13:11:00.242 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.047649191Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-02-07T13:11:00.242 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.048112888Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-02-07T13:11:00.242 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.048624481Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-02-07T13:11:00.242 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.049057375Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-02-07T13:11:00.242 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.049893147Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-02-07T13:11:00.242 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.05088446Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-02-07T13:11:00.242 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.051280629Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-02-07T13:11:00.242 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.05210871Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-02-07T13:11:00.242 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.052506642Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-02-07T13:11:00.243 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.052976055Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-02-07T13:11:00.243 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.053372512Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-07T13:11:00.243 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.055797356Z level=info msg="Executing migration" id="create temp_user v2" 2024-02-07T13:11:00.243 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.056216716Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-02-07T13:11:00.243 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.05869224Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-02-07T13:11:00.243 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.061499484Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-02-07T13:11:00.243 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.064135086Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-02-07T13:11:00.243 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.067124204Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-02-07T13:11:00.243 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.069834323Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-02-07T13:11:00.243 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.073416318Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-02-07T13:11:00.243 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.07629375Z level=info msg="Executing migration" id="create star table" 2024-02-07T13:11:00.243 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.07670214Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-02-07T13:11:00.243 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.079491159Z level=info msg="Executing migration" id="create org table v1" 2024-02-07T13:11:00.243 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.082275144Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-02-07T13:11:00.244 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.085023591Z level=info msg="Executing migration" id="create org_user table v1" 2024-02-07T13:11:00.244 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.08799134Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-02-07T13:11:00.244 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.090815209Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-02-07T13:11:00.244 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.093577866Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-02-07T13:11:00.244 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.094021004Z level=info msg="Executing migration" id="Update org table charset" 2024-02-07T13:11:00.244 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.094348836Z level=info msg="Executing migration" id="Update org_user table charset" 2024-02-07T13:11:00.244 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.097036507Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-02-07T13:11:00.244 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.097400274Z level=info msg="Executing migration" id="create dashboard table" 2024-02-07T13:11:00.244 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.099768979Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-02-07T13:11:00.244 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.10251345Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-02-07T13:11:00.244 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.102976502Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-02-07T13:11:00.244 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.105626584Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-02-07T13:11:00.244 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.10828338Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-07T13:11:00.244 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.111084699Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-02-07T13:11:00.245 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.112195189Z level=info msg="Executing migration" id="create dashboard v2" 2024-02-07T13:11:00.245 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.114845981Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-02-07T13:11:00.245 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.117450263Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-02-07T13:11:00.245 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.120236641Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-02-07T13:11:00.245 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.120708312Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-02-07T13:11:00.245 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.12346973Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-02-07T13:11:00.245 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.126302812Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-02-07T13:11:00.245 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.128904536Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-02-07T13:11:00.245 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.129436269Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-02-07T13:11:00.245 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.130330206Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-02-07T13:11:00.245 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.130829677Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-02-07T13:11:00.245 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.13132367Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-02-07T13:11:00.245 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.132171368Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-02-07T13:11:00.245 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.132600162Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-02-07T13:11:00.246 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.133383089Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-02-07T13:11:00.246 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.133703896Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-02-07T13:11:00.246 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.134219326Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-02-07T13:11:00.246 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.134705663Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-02-07T13:11:00.246 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.13521131Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-02-07T13:11:00.246 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.136279899Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-02-07T13:11:00.246 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.136896657Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-02-07T13:11:00.246 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.137754025Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-02-07T13:11:00.246 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.138162858Z level=info msg="Executing migration" id="Update dashboard title length" 2024-02-07T13:11:00.246 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.139085751Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-07T13:11:00.246 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.139865912Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-02-07T13:11:00.246 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.14069697Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-07T13:11:00.246 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.141620966Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-02-07T13:11:00.246 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.143294085Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-07T13:11:00.246 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.143766194Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-07T13:11:00.247 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.144741524Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-02-07T13:11:00.247 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.145214009Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-02-07T13:11:00.247 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.146095591Z level=info msg="Executing migration" id="Add check_sum column" 2024-02-07T13:11:00.247 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.146908307Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-02-07T13:11:00.247 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.147776584Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-02-07T13:11:00.247 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.148674824Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-02-07T13:11:00.247 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.149434977Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-02-07T13:11:00.247 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.150292496Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-02-07T13:11:00.247 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.151131016Z level=info msg="Executing migration" id="create data_source table" 2024-02-07T13:11:00.247 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.151581826Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-02-07T13:11:00.247 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.152379619Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-02-07T13:11:00.247 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.152835966Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-02-07T13:11:00.247 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.153802269Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-02-07T13:11:00.247 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.154324145Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-02-07T13:11:00.247 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.155463465Z level=info msg="Executing migration" id="create data_source table v2" 2024-02-07T13:11:00.248 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.156289352Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-02-07T13:11:00.248 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.157122496Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-02-07T13:11:00.248 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.157537Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-02-07T13:11:00.248 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.158349965Z level=info msg="Executing migration" id="Add column with_credentials" 2024-02-07T13:11:00.248 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.158876177Z level=info msg="Executing migration" id="Add secure json data column" 2024-02-07T13:11:00.248 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.159444698Z level=info msg="Executing migration" id="Update data_source table charset" 2024-02-07T13:11:00.248 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.160298697Z level=info msg="Executing migration" id="Update initial version to 1" 2024-02-07T13:11:00.248 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.160633713Z level=info msg="Executing migration" id="Add read_only data column" 2024-02-07T13:11:00.248 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.161416113Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-02-07T13:11:00.248 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.162373323Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-02-07T13:11:00.248 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.163114642Z level=info msg="Executing migration" id="Add uid column" 2024-02-07T13:11:00.248 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.164018693Z level=info msg="Executing migration" id="Update uid value" 2024-02-07T13:11:00.248 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.164846776Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-02-07T13:11:00.248 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.165713999Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-02-07T13:11:00.249 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.166689379Z level=info msg="Executing migration" id="create api_key table" 2024-02-07T13:11:00.249 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.16719282Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-02-07T13:11:00.249 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.168049374Z level=info msg="Executing migration" id="add index api_key.key" 2024-02-07T13:11:00.249 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.168937839Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-02-07T13:11:00.249 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.169851483Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-02-07T13:11:00.249 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.170346469Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-02-07T13:11:00.249 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.171204357Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-02-07T13:11:00.249 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.171639161Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-02-07T13:11:00.249 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.172748938Z level=info msg="Executing migration" id="create api_key table v2" 2024-02-07T13:11:00.249 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.17375294Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-02-07T13:11:00.249 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.17447835Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-02-07T13:11:00.249 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.175314466Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-02-07T13:11:00.249 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.176170158Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-02-07T13:11:00.249 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.177260251Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-02-07T13:11:00.249 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.179961961Z level=info msg="Executing migration" id="Update api_key table charset" 2024-02-07T13:11:00.249 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.18268701Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-02-07T13:11:00.250 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.185524841Z level=info msg="Executing migration" id="Add service account foreign key" 2024-02-07T13:11:00.250 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.188287995Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-02-07T13:11:00.250 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.191039724Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-02-07T13:11:00.250 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.193756802Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-02-07T13:11:00.250 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.196460555Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-02-07T13:11:00.250 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.199352267Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-02-07T13:11:00.250 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.202086935Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-02-07T13:11:00.250 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.204993603Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-02-07T13:11:00.250 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.207712357Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-07T13:11:00.250 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.210552988Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-07T13:11:00.250 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.213247794Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-02-07T13:11:00.250 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.215827831Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-02-07T13:11:00.250 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.218493135Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-02-07T13:11:00.250 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.221464467Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-02-07T13:11:00.250 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.224141819Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-07T13:11:00.250 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.226897914Z level=info msg="Executing migration" id="create quota table v1" 2024-02-07T13:11:00.250 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.229677187Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-07T13:11:00.251 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.232224766Z level=info msg="Executing migration" id="Update quota table charset" 2024-02-07T13:11:00.251 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.234854571Z level=info msg="Executing migration" id="create plugin_setting table" 2024-02-07T13:11:00.251 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.237694304Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-07T13:11:00.251 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.240352415Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-02-07T13:11:00.368 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:10:59 smithi191 ceph-mon[27623]: from='client.14541 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-07T13:11:00.490 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.243023126Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-02-07T13:11:00.490 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.245526878Z level=info msg="Executing migration" id="create session table" 2024-02-07T13:11:00.490 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.248136266Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-02-07T13:11:00.490 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.250859545Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-02-07T13:11:00.490 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.253689793Z level=info msg="Executing migration" id="create playlist table v2" 2024-02-07T13:11:00.490 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.256370039Z level=info msg="Executing migration" id="create playlist item table v2" 2024-02-07T13:11:00.490 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.259992317Z level=info msg="Executing migration" id="Update playlist table charset" 2024-02-07T13:11:00.491 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.261782788Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-02-07T13:11:00.491 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.264641738Z level=info msg="Executing migration" id="drop preferences table v2" 2024-02-07T13:11:00.491 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.267118043Z level=info msg="Executing migration" id="drop preferences table v3" 2024-02-07T13:11:00.491 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.269835846Z level=info msg="Executing migration" id="create preferences table v3" 2024-02-07T13:11:00.491 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.272691368Z level=info msg="Executing migration" id="Update preferences table charset" 2024-02-07T13:11:00.491 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.273366492Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-02-07T13:11:00.491 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.274250256Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-02-07T13:11:00.491 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.275102432Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-02-07T13:11:00.491 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.275971388Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-02-07T13:11:00.491 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.276810902Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-02-07T13:11:00.491 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.277791705Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-02-07T13:11:00.491 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.278453442Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-02-07T13:11:00.491 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.279464917Z level=info msg="Executing migration" id="create alert table v1" 2024-02-07T13:11:00.492 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.280144176Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-02-07T13:11:00.492 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.281034587Z level=info msg="Executing migration" id="add index alert state" 2024-02-07T13:11:00.492 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.282031668Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-02-07T13:11:00.492 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.282751281Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-02-07T13:11:00.492 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.283740854Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-07T13:11:00.492 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.284362747Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-07T13:11:00.492 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.285310403Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-07T13:11:00.492 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.28692814Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-02-07T13:11:00.492 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.287740924Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-07T13:11:00.492 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.288595433Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-02-07T13:11:00.492 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.289519951Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-02-07T13:11:00.492 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.290765298Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-02-07T13:11:00.492 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.291618093Z level=info msg="Executing migration" id="Add column is_default" 2024-02-07T13:11:00.492 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.292437449Z level=info msg="Executing migration" id="Add column frequency" 2024-02-07T13:11:00.493 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.293324998Z level=info msg="Executing migration" id="Add column send_reminder" 2024-02-07T13:11:00.493 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.294149717Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-02-07T13:11:00.493 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.29516772Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-02-07T13:11:00.493 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.295844846Z level=info msg="Executing migration" id="Update alert table charset" 2024-02-07T13:11:00.493 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.296828637Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-02-07T13:11:00.493 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.297587604Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-02-07T13:11:00.493 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.298589689Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-07T13:11:00.493 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.299288085Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-02-07T13:11:00.493 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.300316798Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-02-07T13:11:00.493 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.301032003Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-07T13:11:00.493 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.302067851Z level=info msg="Executing migration" id="Add for to alert table" 2024-02-07T13:11:00.493 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.302808223Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-02-07T13:11:00.493 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.303672482Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-02-07T13:11:00.493 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.307439639Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-02-07T13:11:00.494 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.310195881Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-02-07T13:11:00.494 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.313066164Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-02-07T13:11:00.494 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.315991936Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-02-07T13:11:00.494 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.318681684Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-02-07T13:11:00.494 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.321440408Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-02-07T13:11:00.494 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.324242496Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-02-07T13:11:00.494 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.327235004Z level=info msg="Executing migration" id="create annotation table v5" 2024-02-07T13:11:00.494 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.330033802Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-02-07T13:11:00.494 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.332738311Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-02-07T13:11:00.495 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.335525238Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-02-07T13:11:00.495 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.338248149Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-02-07T13:11:00.495 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.341031939Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-02-07T13:11:00.495 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.343727069Z level=info msg="Executing migration" id="Update annotation table charset" 2024-02-07T13:11:00.495 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.346504669Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-02-07T13:11:00.495 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.34943153Z level=info msg="Executing migration" id="Drop category_id index" 2024-02-07T13:11:00.496 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.352180724Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-02-07T13:11:00.496 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.355037087Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-02-07T13:11:00.496 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.357535592Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-07T13:11:00.496 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.360243717Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-07T13:11:00.496 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.362839924Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-07T13:11:00.496 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.365699316Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-02-07T13:11:00.496 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.368299574Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-07T13:11:00.496 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.370918635Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-02-07T13:11:00.496 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.373647838Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-02-07T13:11:00.496 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.376429348Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-02-07T13:11:00.496 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.379158814Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-02-07T13:11:00.496 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.381819173Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-02-07T13:11:00.496 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.384457345Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-02-07T13:11:00.497 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.387185195Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-02-07T13:11:00.497 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.387654527Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-02-07T13:11:00.497 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.388474262Z level=info msg="Executing migration" id="Add epoch_end column" 2024-02-07T13:11:00.497 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.389187708Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-02-07T13:11:00.497 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.389672906Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-02-07T13:11:00.497 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.390476715Z level=info msg="Executing migration" id="Move region to single row" 2024-02-07T13:11:00.497 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.390885889Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-02-07T13:11:00.497 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.391354554Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-07T13:11:00.497 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.392221487Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-07T13:11:00.497 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.395194014Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-07T13:11:00.497 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.397990805Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-07T13:11:00.497 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.400759682Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-02-07T13:11:00.497 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.403340633Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-02-07T13:11:00.497 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.406276757Z level=info msg="Executing migration" id="create test_data table" 2024-02-07T13:11:00.498 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.406843127Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-02-07T13:11:00.498 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.409189936Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-02-07T13:11:00.498 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.411764278Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-07T13:11:00.498 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.414637416Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-02-07T13:11:00.498 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.417338422Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-02-07T13:11:00.498 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.420028069Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-02-07T13:11:00.498 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.42287365Z level=info msg="Executing migration" id="create team table" 2024-02-07T13:11:00.498 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.427452729Z level=info msg="Executing migration" id="add index team.org_id" 2024-02-07T13:11:00.498 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.428245457Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-02-07T13:11:00.498 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.431052976Z level=info msg="Executing migration" id="create team member table" 2024-02-07T13:11:00.498 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.433872867Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-02-07T13:11:00.498 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.436708614Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-02-07T13:11:00.498 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.437178703Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-02-07T13:11:00.498 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.440162191Z level=info msg="Executing migration" id="Add column email to team table" 2024-02-07T13:11:00.498 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.442472072Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-02-07T13:11:00.499 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.445160023Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-02-07T13:11:00.499 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.446235323Z level=info msg="Executing migration" id="create dashboard acl table" 2024-02-07T13:11:00.499 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.448831895Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-02-07T13:11:00.499 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.451573947Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-07T13:11:00.499 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.452083329Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-07T13:11:00.499 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.454272648Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-02-07T13:11:00.499 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.457173826Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-02-07T13:11:00.499 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.459772784Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-02-07T13:11:00.499 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.460355952Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-02-07T13:11:00.499 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.460795154Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-02-07T13:11:00.499 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.461220789Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-02-07T13:11:00.499 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.461557725Z level=info msg="Executing migration" id="create tag table" 2024-02-07T13:11:00.499 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.461955698Z level=info msg="Executing migration" id="add index tag.key_value" 2024-02-07T13:11:00.499 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.462405845Z level=info msg="Executing migration" id="create login attempt table" 2024-02-07T13:11:00.500 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.462798871Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-02-07T13:11:00.500 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.463279758Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-02-07T13:11:00.500 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.463702517Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-07T13:11:00.500 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.46560659Z level=info msg="Executing migration" id="create login_attempt v2" 2024-02-07T13:11:00.500 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.466038024Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-02-07T13:11:00.500 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.46648319Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-02-07T13:11:00.500 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.466795858Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-02-07T13:11:00.500 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.467179595Z level=info msg="Executing migration" id="create user auth table" 2024-02-07T13:11:00.500 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.467581937Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-07T13:11:00.500 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.468023356Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-02-07T13:11:00.500 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.468340679Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-02-07T13:11:00.500 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.469089178Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-02-07T13:11:00.500 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.469820944Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-02-07T13:11:00.500 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.470599322Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-02-07T13:11:00.501 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.47137417Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-02-07T13:11:00.501 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.471818872Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-02-07T13:11:00.501 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.472577392Z level=info msg="Executing migration" id="create server_lock table" 2024-02-07T13:11:00.501 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.473087478Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-02-07T13:11:00.501 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.473547406Z level=info msg="Executing migration" id="create user auth token table" 2024-02-07T13:11:00.501 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.47399191Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-02-07T13:11:00.501 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.476704415Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-02-07T13:11:00.501 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.477205186Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-02-07T13:11:00.501 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.477646792Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-02-07T13:11:00.501 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.478447044Z level=info msg="Executing migration" id="create cache_data table" 2024-02-07T13:11:00.501 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.478863753Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-02-07T13:11:00.501 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.479313234Z level=info msg="Executing migration" id="create short_url table v1" 2024-02-07T13:11:00.501 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.47974367Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-02-07T13:11:00.501 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.480223356Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-02-07T13:11:00.501 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.480552203Z level=info msg="Executing migration" id="delete alert_definition table" 2024-02-07T13:11:00.502 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.480841361Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-02-07T13:11:00.502 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.48225036Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-02-07T13:11:00.502 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.485021025Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-02-07T13:11:00.502 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.485463224Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-02-07T13:11:00.502 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.485739909Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-02-07T13:11:00.502 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.486187803Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-02-07T13:11:00.502 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.486619313Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-02-07T13:11:00.502 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.489269854Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-02-07T13:11:00.502 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.489724468Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-02-07T13:11:00.502 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.490530633Z level=info msg="Executing migration" id="drop alert_definition table" 2024-02-07T13:11:00.502 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.49196526Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-02-07T13:11:00.746 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.494772377Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-02-07T13:11:00.746 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.495328019Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-07T13:11:00.746 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.497850166Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-07T13:11:00.746 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.498302154Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-07T13:11:00.746 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.500552066Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-02-07T13:11:00.746 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.501007816Z level=info msg="Executing migration" id="create alert_instance table" 2024-02-07T13:11:00.746 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.502027111Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-07T13:11:00.746 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.502753945Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-07T13:11:00.746 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.503522771Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-02-07T13:11:00.746 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.504374897Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-07T13:11:00.747 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.504813137Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-02-07T13:11:00.747 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.505599613Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-02-07T13:11:00.747 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.509026897Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-02-07T13:11:00.747 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.512373846Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-07T13:11:00.747 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.512842501Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-02-07T13:11:00.747 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.513808868Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-02-07T13:11:00.747 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.514768081Z level=info msg="Executing migration" id="create alert_rule table" 2024-02-07T13:11:00.748 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.515222056Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-02-07T13:11:00.748 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.516209221Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-02-07T13:11:00.748 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.516938197Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-07T13:11:00.748 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.517771554Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-02-07T13:11:00.748 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.51870495Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-02-07T13:11:00.748 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.519599947Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-02-07T13:11:00.748 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.520474892Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-02-07T13:11:00.748 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.52132851Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-02-07T13:11:00.748 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.52215345Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-07T13:11:00.749 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.523139022Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-02-07T13:11:00.749 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.524053718Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-02-07T13:11:00.749 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.524921564Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-07T13:11:00.749 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.525901826Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-02-07T13:11:00.749 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.526832092Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-02-07T13:11:00.749 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.527673255Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-02-07T13:11:00.749 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.528586746Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-02-07T13:11:00.749 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.529461713Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-07T13:11:00.749 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.530334697Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-07T13:11:00.749 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.530795161Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-07T13:11:00.749 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.531154242Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-02-07T13:11:00.751 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.532026303Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-02-07T13:11:00.751 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.532873228Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-02-07T13:11:00.751 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.533751088Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-02-07T13:11:00.751 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.534594857Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-02-07T13:11:00.751 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.535460348Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-02-07T13:11:00.751 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.536395271Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-02-07T13:11:00.751 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.537172542Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-02-07T13:11:00.751 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.538037342Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-07T13:11:00.752 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.538994652Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-02-07T13:11:00.752 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.540041227Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-02-07T13:11:00.752 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.543375335Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-02-07T13:11:00.752 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.545828857Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-02-07T13:11:00.752 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.548671241Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-02-07T13:11:00.752 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.551253934Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-02-07T13:11:00.752 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.554118695Z level=info msg="Executing migration" id="create provenance_type table" 2024-02-07T13:11:00.752 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.554766961Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-02-07T13:11:00.752 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.557697191Z level=info msg="Executing migration" id="create alert_image table" 2024-02-07T13:11:00.752 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.560357309Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-02-07T13:11:00.752 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.562906427Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-02-07T13:11:00.752 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.565741775Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-02-07T13:11:00.752 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.568374802Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-02-07T13:11:00.753 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.571077152Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-02-07T13:11:00.753 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.571177825Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-02-07T13:11:00.753 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.571479066Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-02-07T13:11:00.753 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.574102725Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-02-07T13:11:00.753 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.574564387Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-02-07T13:11:00.753 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.574696816Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-02-07T13:11:00.753 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.574847693Z level=info msg="alerts found to migrate" alerts=0 2024-02-07T13:11:00.753 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.57690904Z level=info msg="Executing migration" id="create library_element table v1" 2024-02-07T13:11:00.753 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.577874459Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-02-07T13:11:00.753 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.578496058Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-02-07T13:11:00.753 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.579284353Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-02-07T13:11:00.753 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.580069449Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-02-07T13:11:00.753 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.580878939Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-02-07T13:11:00.755 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.581211117Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-02-07T13:11:00.755 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.58212799Z level=info msg="Executing migration" id="create data_keys table" 2024-02-07T13:11:00.755 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.582742041Z level=info msg="Executing migration" id="create secrets table" 2024-02-07T13:11:00.755 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.583200738Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-02-07T13:11:00.755 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.587706825Z level=info msg="Executing migration" id="add name column into data_keys" 2024-02-07T13:11:00.755 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.588675009Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-02-07T13:11:00.755 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.589517928Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-02-07T13:11:00.755 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.593997783Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-02-07T13:11:00.756 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.598270993Z level=info msg="Executing migration" id="create kv_store table v1" 2024-02-07T13:11:00.756 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.598716088Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-02-07T13:11:00.756 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.599544122Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-02-07T13:11:00.756 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.600391901Z level=info msg="Executing migration" id="create permission table" 2024-02-07T13:11:00.756 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.601248439Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-02-07T13:11:00.756 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.601710354Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-02-07T13:11:00.756 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.602596269Z level=info msg="Executing migration" id="create role table" 2024-02-07T13:11:00.756 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.603449312Z level=info msg="Executing migration" id="add column display_name" 2024-02-07T13:11:00.756 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.604419873Z level=info msg="Executing migration" id="add column group_name" 2024-02-07T13:11:00.756 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.605390311Z level=info msg="Executing migration" id="add index role.org_id" 2024-02-07T13:11:00.756 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.606248475Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-02-07T13:11:00.756 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.607091605Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-02-07T13:11:00.756 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.607941713Z level=info msg="Executing migration" id="create team role table" 2024-02-07T13:11:00.756 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.608368349Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-02-07T13:11:00.757 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.60922136Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-02-07T13:11:00.757 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.610073479Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-02-07T13:11:00.757 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.610918864Z level=info msg="Executing migration" id="create user role table" 2024-02-07T13:11:00.757 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.611340172Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-02-07T13:11:00.757 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.612191641Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-02-07T13:11:00.757 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.613064051Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-02-07T13:11:00.757 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.613881175Z level=info msg="Executing migration" id="create builtin role table" 2024-02-07T13:11:00.757 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.614337631Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-02-07T13:11:00.757 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.617081737Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-02-07T13:11:00.757 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.619826428Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-02-07T13:11:00.757 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.622572446Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-02-07T13:11:00.757 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.623132501Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-02-07T13:11:00.757 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.633077392Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-02-07T13:11:00.757 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.633552899Z level=info msg="Executing migration" id="add unique index role.uid" 2024-02-07T13:11:00.758 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.63620349Z level=info msg="Executing migration" id="create seed assignment table" 2024-02-07T13:11:00.758 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.636628345Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-02-07T13:11:00.758 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.637097335Z level=info msg="Executing migration" id="add column hidden to role table" 2024-02-07T13:11:00.758 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.639858638Z level=info msg="Executing migration" id="create query_history table v1" 2024-02-07T13:11:00.758 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.642676343Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-02-07T13:11:00.758 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.645511936Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-02-07T13:11:00.758 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.645834457Z level=info msg="Executing migration" id="teams permissions migration" 2024-02-07T13:11:00.758 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.648413036Z level=info msg="Executing migration" id="dashboard permissions" 2024-02-07T13:11:00.758 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.65117582Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-02-07T13:11:00.758 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.653911416Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-02-07T13:11:00.758 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.654260731Z level=info msg="Executing migration" id="alerting notification permissions" 2024-02-07T13:11:00.758 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.657046441Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-02-07T13:11:00.759 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.65748826Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-02-07T13:11:00.759 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.659746837Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-02-07T13:11:00.759 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.662298796Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-02-07T13:11:00.759 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.662651526Z level=info msg="Executing migration" id="create correlation table v1" 2024-02-07T13:11:00.759 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.665427837Z level=info msg="Executing migration" id="add index correlations.uid" 2024-02-07T13:11:00.759 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.66804279Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-02-07T13:11:00.759 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.670535606Z level=info msg="Executing migration" id="add correlation config column" 2024-02-07T13:11:00.759 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.671557581Z level=info msg="Executing migration" id="create entity_events table" 2024-02-07T13:11:00.759 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.674253242Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-02-07T13:11:00.759 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.676903994Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-07T13:11:00.759 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.677036279Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-07T13:11:00.759 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.679680222Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-07T13:11:00.759 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.679786446Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-07T13:11:00.760 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.680099958Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-02-07T13:11:00.760 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.682787543Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-02-07T13:11:00.760 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.685284967Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-02-07T13:11:00.760 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.68789341Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-07T13:11:00.760 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.688379849Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-02-07T13:11:00.760 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.691011646Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-07T13:11:00.760 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.691477428Z level=info msg="Executing migration" id="Drop public config table" 2024-02-07T13:11:00.760 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.69188253Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-02-07T13:11:00.760 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.69234707Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-02-07T13:11:00.760 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.692802045Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-07T13:11:00.760 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.693303744Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-02-07T13:11:00.760 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.69382326Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-02-07T13:11:00.760 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.697349348Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-02-07T13:11:00.760 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.698462263Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-02-07T13:11:00.761 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.699470234Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-02-07T13:11:00.761 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.69980099Z level=info msg="Executing migration" id="add share column" 2024-02-07T13:11:00.761 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.700799571Z level=info msg="Executing migration" id="create default alerting folders" 2024-02-07T13:11:00.761 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.701622772Z level=info msg="Executing migration" id="create file table" 2024-02-07T13:11:00.761 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.702399564Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-02-07T13:11:00.761 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.702912171Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-02-07T13:11:00.761 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.703715228Z level=info msg="Executing migration" id="create file_meta table" 2024-02-07T13:11:00.761 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.704601669Z level=info msg="Executing migration" id="file table idx: path key" 2024-02-07T13:11:00.761 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.705430185Z level=info msg="Executing migration" id="set path collation in file table" 2024-02-07T13:11:00.761 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.705859368Z level=info msg="Executing migration" id="managed permissions migration" 2024-02-07T13:11:00.761 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.706711155Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-02-07T13:11:00.761 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.707573588Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-02-07T13:11:00.762 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.708409664Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-02-07T13:11:00.762 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.71016156Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-02-07T13:11:00.762 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.711055143Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-02-07T13:11:00.762 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.711869463Z level=info msg="Executing migration" id="update group index for alert rules" 2024-02-07T13:11:00.762 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.712693667Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-02-07T13:11:00.762 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.713180326Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-02-07T13:11:00.762 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.714038889Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-02-07T13:11:00.762 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.715770788Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-02-07T13:11:00.762 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.71744348Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-02-07T13:11:00.762 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.718248478Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-02-07T13:11:00.762 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.737375354Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-02-07T13:11:00.762 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.738183273Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-02-07T13:11:00.762 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.739024987Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-02-07T13:11:00.762 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.745420983Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-02-07T13:11:00.762 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.74623705Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-02-07T13:11:00.763 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.749683604Z level=info msg="Executing migration" id="create folder table" 2024-02-07T13:11:00.906 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:00 smithi067 ceph-mon[25698]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:00.906 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:00 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:00.906 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:00 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:00.906 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:00 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:00.906 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:00 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:01.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-mon[27461]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:01.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:01.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:01.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:01.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:01.194 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.751683545Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-02-07T13:11:01.194 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.754559529Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-02-07T13:11:01.194 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.757267783Z level=info msg="Executing migration" id="Update folder title length" 2024-02-07T13:11:01.194 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.760064972Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-02-07T13:11:01.194 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=migrator t=2024-02-07T13:11:00.762899659Z level=info msg="migrations completed" performed=485 skipped=0 duration=725.661763ms 2024-02-07T13:11:01.194 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=sqlstore t=2024-02-07T13:11:00.763447352Z level=info msg="Created default organization" 2024-02-07T13:11:01.194 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=plugin.loader t=2024-02-07T13:11:00.832402162Z level=info msg="Plugin registered" pluginID=input 2024-02-07T13:11:01.194 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=plugin.loader t=2024-02-07T13:11:00.86677999Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-02-07T13:11:01.194 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=plugin.loader t=2024-02-07T13:11:00.86702912Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-02-07T13:11:01.194 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=secrets t=2024-02-07T13:11:00.867084264Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-02-07T13:11:01.194 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=query_data t=2024-02-07T13:11:00.869564989Z level=info msg="Query Service initialization" 2024-02-07T13:11:01.195 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:00 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=live.push_http t=2024-02-07T13:11:00.872450972Z level=info msg="Live Push Gateway initialization" 2024-02-07T13:11:01.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:00 smithi191 ceph-mon[27623]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:01.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:00 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:01.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:00 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:01.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:00 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:01.370 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:00 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:01.545 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:01 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: ::ffff:172.21.15.159 - - [07/Feb/2024:13:11:01] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-07T13:11:01.619 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-02-07T13:11:01.619 INFO:teuthology.orchestra.run.smithi067.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-07T13:11:01.827 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:01 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=infra.usagestats.collector t=2024-02-07T13:11:01.571496161Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-02-07T13:11:01.827 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:01 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=provisioning.datasources t=2024-02-07T13:11:01.572445605Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-07T13:11:01.827 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:01 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=provisioning.datasources t=2024-02-07T13:11:01.583418532Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-02-07T13:11:01.827 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:01 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=provisioning.alerting t=2024-02-07T13:11:01.594794906Z level=info msg="starting to provision alerting" 2024-02-07T13:11:01.827 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:01 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=provisioning.alerting t=2024-02-07T13:11:01.5948231Z level=info msg="finished to provision alerting" 2024-02-07T13:11:01.828 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:01 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=ngalert.state.manager t=2024-02-07T13:11:01.594939293Z level=info msg="Warming state cache for startup" 2024-02-07T13:11:01.828 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:01 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=grafanaStorageLogger t=2024-02-07T13:11:01.595130864Z level=info msg="storage starting" 2024-02-07T13:11:01.828 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:01 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=http.server t=2024-02-07T13:11:01.597704877Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-02-07T13:11:01.828 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:01 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=ngalert.state.manager t=2024-02-07T13:11:01.601055276Z level=info msg="State cache has been initialized" states=0 duration=6.114459ms 2024-02-07T13:11:01.828 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:01 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=ngalert.multiorg.alertmanager t=2024-02-07T13:11:01.601106119Z level=info msg="starting MultiOrg Alertmanager" 2024-02-07T13:11:01.828 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:01 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=ticker t=2024-02-07T13:11:01.601129843Z level=info msg=starting first_tick=2024-02-07T13:11:10Z 2024-02-07T13:11:01.828 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:11:01 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a[38245]: logger=sqlstore.transactions t=2024-02-07T13:11:01.607771717Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-07T13:11:02.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:01 smithi159 ceph-mon[27461]: Deploying daemon node-exporter.a on smithi067 2024-02-07T13:11:02.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:01 smithi159 ceph-mon[27461]: from='client.? 172.21.15.67:0/1924556599' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-07T13:11:02.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:01 smithi067 ceph-mon[25698]: Deploying daemon node-exporter.a on smithi067 2024-02-07T13:11:02.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:01 smithi067 ceph-mon[25698]: from='client.? 172.21.15.67:0/1924556599' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-07T13:11:02.368 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:01 smithi191 ceph-mon[27623]: Deploying daemon node-exporter.a on smithi067 2024-02-07T13:11:02.368 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:01 smithi191 ceph-mon[27623]: from='client.? 172.21.15.67:0/1924556599' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-07T13:11:02.492 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-07T13:11:02.492 INFO:tasks.cephadm:Setup complete, yielding 2024-02-07T13:11:02.493 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-07T13:11:02.503 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi067.front.sepia.ceph.com 2024-02-07T13:11:02.503 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- bash -c 'stat -c '"'"'%u %g'"'"' /var/log/ceph | grep '"'"'167 167'"'"'' 2024-02-07T13:11:02.744 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:11:03.119 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:02 smithi067 ceph-mon[25698]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:03.119 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:02 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:03.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:02 smithi159 ceph-mon[27461]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:03.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:02 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:03.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:02 smithi191 ceph-mon[27623]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:03.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:02 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:04.306 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:04 smithi067 bash[49604]: Writing manifest to image destination 2024-02-07T13:11:04.893 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:04 smithi067 podman[49604]: 2024-02-07 13:11:04.55807093 +0000 UTC m=+3.114954060 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-07T13:11:04.894 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:04 smithi067 podman[49604]: 2024-02-07 13:11:04.790020933 +0000 UTC m=+3.346904062 container create 81c9b873491c45336fa5fc877c9b4ec223af1fcf1c15629bc3bca9dda59ea34c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-07T13:11:05.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:04 smithi159 ceph-mon[27461]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:05.194 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:04 smithi067 ceph-mon[25698]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:05.368 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:04 smithi191 ceph-mon[27623]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:05.386 INFO:teuthology.orchestra.run.smithi067.stdout:167 167 2024-02-07T13:11:05.556 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 podman[49604]: 2024-02-07 13:11:05.194651207 +0000 UTC m=+3.751534338 container init 81c9b873491c45336fa5fc877c9b4ec223af1fcf1c15629bc3bca9dda59ea34c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-07T13:11:05.556 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 podman[49604]: 2024-02-07 13:11:05.201620672 +0000 UTC m=+3.758503806 container start 81c9b873491c45336fa5fc877c9b4ec223af1fcf1c15629bc3bca9dda59ea34c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-07T13:11:05.556 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.205Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-07T13:11:05.556 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.205Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-07T13:11:05.556 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.205Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-07T13:11:05.556 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.205Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-07T13:11:05.557 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-07T13:11:05.557 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-07T13:11:05.557 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-07T13:11:05.557 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=arp 2024-02-07T13:11:05.557 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-07T13:11:05.557 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-07T13:11:05.557 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-07T13:11:05.557 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-07T13:11:05.557 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-07T13:11:05.557 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-07T13:11:05.558 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-07T13:11:05.558 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-07T13:11:05.558 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=edac 2024-02-07T13:11:05.558 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-07T13:11:05.558 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-07T13:11:05.558 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-07T13:11:05.558 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-07T13:11:05.558 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-07T13:11:05.558 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-07T13:11:05.558 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-07T13:11:05.558 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-07T13:11:05.558 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-07T13:11:05.559 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-07T13:11:05.559 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-07T13:11:05.559 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-07T13:11:05.559 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-07T13:11:05.559 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-07T13:11:05.559 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-07T13:11:05.559 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-07T13:11:05.559 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=os 2024-02-07T13:11:05.559 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-07T13:11:05.559 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-07T13:11:05.559 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-07T13:11:05.559 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-07T13:11:05.559 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-07T13:11:05.559 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-07T13:11:05.559 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-07T13:11:05.560 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=stat 2024-02-07T13:11:05.560 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-07T13:11:05.560 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-07T13:11:05.560 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-07T13:11:05.560 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=time 2024-02-07T13:11:05.560 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-07T13:11:05.560 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=uname 2024-02-07T13:11:05.560 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-07T13:11:05.560 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-07T13:11:05.560 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-07T13:11:05.560 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-07T13:11:05.560 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a[49818]: ts=2024-02-07T13:11:05.206Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-07T13:11:06.055 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 bash[49604]: 81c9b873491c45336fa5fc877c9b4ec223af1fcf1c15629bc3bca9dda59ea34c 2024-02-07T13:11:06.056 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:11:05 smithi067 systemd[1]: Started Ceph node-exporter.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:11:06.176 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- bash -c 'ceph orch status' 2024-02-07T13:11:06.422 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:11:06.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:06 smithi159 ceph-mon[27461]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:06.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:06 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:06.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:06 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:06.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:06 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:06.943 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:06 smithi159 ceph-mon[27461]: Deploying daemon node-exporter.b on smithi159 2024-02-07T13:11:07.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:06 smithi067 ceph-mon[25698]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:07.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:06 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:07.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:06 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:07.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:06 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:07.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:06 smithi067 ceph-mon[25698]: Deploying daemon node-exporter.b on smithi159 2024-02-07T13:11:07.118 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:06 smithi191 ceph-mon[27623]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:07.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:06 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:07.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:06 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:07.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:06 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:07.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:06 smithi191 ceph-mon[27623]: Deploying daemon node-exporter.b on smithi159 2024-02-07T13:11:07.443 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:06 smithi159 bash[39266]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-07T13:11:07.943 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:07 smithi159 bash[39266]: Getting image source signatures 2024-02-07T13:11:07.943 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:07 smithi159 bash[39266]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-07T13:11:07.943 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:07 smithi159 bash[39266]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-07T13:11:07.943 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:07 smithi159 bash[39266]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-07T13:11:08.619 INFO:teuthology.orchestra.run.smithi067.stdout:Backend: cephadm 2024-02-07T13:11:08.619 INFO:teuthology.orchestra.run.smithi067.stdout:Available: Yes 2024-02-07T13:11:08.619 INFO:teuthology.orchestra.run.smithi067.stdout:Paused: No 2024-02-07T13:11:09.017 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:08 smithi067 ceph-mon[25698]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:09.118 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:08 smithi191 ceph-mon[27623]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:09.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:08 smithi159 ceph-mon[27461]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:09.193 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:08 smithi159 bash[39266]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-07T13:11:09.193 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:08 smithi159 bash[39266]: Writing manifest to image destination 2024-02-07T13:11:09.407 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-07T13:11:09.617 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 podman[39266]: 2024-02-07 13:11:09.208325741 +0000 UTC m=+2.350664052 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-07T13:11:09.617 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 podman[39266]: 2024-02-07 13:11:09.306658668 +0000 UTC m=+2.448996973 container create 5030b8182be2fa8900cd73317de7c992387473fd74f6381a22924e92211332cd (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-07T13:11:09.617 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 podman[39266]: 2024-02-07 13:11:09.516658 +0000 UTC m=+2.658996303 container init 5030b8182be2fa8900cd73317de7c992387473fd74f6381a22924e92211332cd (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-07T13:11:09.617 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 podman[39266]: 2024-02-07 13:11:09.523378854 +0000 UTC m=+2.665717159 container start 5030b8182be2fa8900cd73317de7c992387473fd74f6381a22924e92211332cd (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-07T13:11:09.617 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.527Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-07T13:11:09.617 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.527Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-07T13:11:09.617 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.527Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-07T13:11:09.617 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-07T13:11:09.617 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-07T13:11:09.618 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-07T13:11:09.618 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-07T13:11:09.618 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=arp 2024-02-07T13:11:09.618 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-07T13:11:09.618 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-07T13:11:09.618 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-07T13:11:09.618 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-07T13:11:09.619 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-07T13:11:09.619 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-07T13:11:09.619 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-07T13:11:09.619 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-07T13:11:09.619 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=edac 2024-02-07T13:11:09.619 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-07T13:11:09.619 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-07T13:11:09.619 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-07T13:11:09.619 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-07T13:11:09.619 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-07T13:11:09.619 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-07T13:11:09.619 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-07T13:11:09.619 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-07T13:11:09.619 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-07T13:11:09.619 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-07T13:11:09.620 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-07T13:11:09.620 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-07T13:11:09.620 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-07T13:11:09.620 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-07T13:11:09.620 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-07T13:11:09.620 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-07T13:11:09.620 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=os 2024-02-07T13:11:09.620 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-07T13:11:09.620 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-07T13:11:09.620 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-07T13:11:09.620 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-07T13:11:09.620 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-07T13:11:09.620 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-07T13:11:09.620 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-07T13:11:09.620 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=stat 2024-02-07T13:11:09.621 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-07T13:11:09.621 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-07T13:11:09.621 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-07T13:11:09.621 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=time 2024-02-07T13:11:09.621 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-07T13:11:09.621 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=uname 2024-02-07T13:11:09.621 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-07T13:11:09.621 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-07T13:11:09.621 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.528Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-07T13:11:09.621 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.529Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-07T13:11:09.621 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b[39317]: ts=2024-02-07T13:11:09.529Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-07T13:11:09.621 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 bash[39266]: 5030b8182be2fa8900cd73317de7c992387473fd74f6381a22924e92211332cd 2024-02-07T13:11:09.876 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:11:09 smithi159 systemd[1]: Started Ceph node-exporter.b for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:11:10.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:09 smithi067 ceph-mon[25698]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:11:10.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:09 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:10.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:09 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:10.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:09 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:10.118 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:09 smithi191 ceph-mon[27623]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:11:10.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:09 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:10.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:09 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:10.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:09 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:10.131 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:11:10.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-mon[27461]: from='client.14553 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:11:10.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:10.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:10.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:09 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:11.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:10 smithi067 ceph-mon[25698]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:11.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:10 smithi067 ceph-mon[25698]: Deploying daemon node-exporter.c on smithi191 2024-02-07T13:11:11.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:10 smithi159 ceph-mon[27461]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:11.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:10 smithi159 ceph-mon[27461]: Deploying daemon node-exporter.c on smithi191 2024-02-07T13:11:11.348 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:10 smithi191 ceph-mon[27623]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:11.348 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:10 smithi191 ceph-mon[27623]: Deploying daemon node-exporter.c on smithi191 2024-02-07T13:11:11.439 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:11 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: ::ffff:172.21.15.159 - - [07/Feb/2024:13:11:11] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-07T13:11:11.869 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:11 smithi191 bash[39114]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-07T13:11:12.618 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:12 smithi191 bash[39114]: Getting image source signatures 2024-02-07T13:11:12.618 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:12 smithi191 bash[39114]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-07T13:11:12.619 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:12 smithi191 bash[39114]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-07T13:11:12.619 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:12 smithi191 bash[39114]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-07T13:11:13.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:12 smithi067 ceph-mon[25698]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:13.055 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:12 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:11:13.102 INFO:teuthology.orchestra.run.smithi067.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-07T13:11:13.103 INFO:teuthology.orchestra.run.smithi067.stdout:agent.smithi067 smithi067 running 53s ago 3m - - 2024-02-07T13:11:13.103 INFO:teuthology.orchestra.run.smithi067.stdout:agent.smithi159 smithi159 running 65s ago 3m - - 2024-02-07T13:11:13.103 INFO:teuthology.orchestra.run.smithi067.stdout:agent.smithi191 smithi191 running 63s ago 3m - - 2024-02-07T13:11:13.103 INFO:teuthology.orchestra.run.smithi067.stdout:alertmanager.a smithi067 *:9093,9094 starting - - - - 2024-02-07T13:11:13.103 INFO:teuthology.orchestra.run.smithi067.stdout:grafana.a smithi159 *:3000 starting - - - - 2024-02-07T13:11:13.103 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.x smithi159 *:8443,8765 running (2m) 65s ago 2m 448M - 19.0.0-1252-g44458cce 79957b372307 86d280d6c1bf 2024-02-07T13:11:13.103 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.y smithi067 *:9283,8765 running (4m) 53s ago 4m 512M - 19.0.0-1252-g44458cce 79957b372307 c67bb53a16c7 2024-02-07T13:11:13.103 INFO:teuthology.orchestra.run.smithi067.stdout:mgr.z smithi191 *:8443,8765 running (2m) 63s ago 2m 448M - 19.0.0-1252-g44458cce 79957b372307 b66654ac8d28 2024-02-07T13:11:13.103 INFO:teuthology.orchestra.run.smithi067.stdout:mon.a smithi067 running (4m) 53s ago 5m 46.9M 2048M 19.0.0-1252-g44458cce 79957b372307 4aae93977019 2024-02-07T13:11:13.103 INFO:teuthology.orchestra.run.smithi067.stdout:mon.b smithi159 running (3m) 65s ago 3m 40.8M 2048M 19.0.0-1252-g44458cce 79957b372307 fe28fbc6f0a8 2024-02-07T13:11:13.103 INFO:teuthology.orchestra.run.smithi067.stdout:mon.c smithi191 running (3m) 63s ago 3m 44.0M 2048M 19.0.0-1252-g44458cce 79957b372307 f1e3ce748477 2024-02-07T13:11:13.103 INFO:teuthology.orchestra.run.smithi067.stdout:node-exporter.a smithi067 *:9100 starting - - - - 2024-02-07T13:11:13.103 INFO:teuthology.orchestra.run.smithi067.stdout:node-exporter.b smithi159 *:9100 starting - - - - 2024-02-07T13:11:13.104 INFO:teuthology.orchestra.run.smithi067.stdout:osd.0 smithi067 running (2m) 53s ago 2m 49.7M 4096M 19.0.0-1252-g44458cce 79957b372307 512fdcca12ba 2024-02-07T13:11:13.104 INFO:teuthology.orchestra.run.smithi067.stdout:osd.1 smithi159 running (2m) 65s ago 2m 59.0M 4096M 19.0.0-1252-g44458cce 79957b372307 d162295b5a5d 2024-02-07T13:11:13.104 INFO:teuthology.orchestra.run.smithi067.stdout:osd.2 smithi191 running (96s) 63s ago 95s 61.2M 4096M 19.0.0-1252-g44458cce 79957b372307 d351f9fcee14 2024-02-07T13:11:13.104 INFO:teuthology.orchestra.run.smithi067.stdout:prometheus.a smithi159 *:9095 starting - - - - 2024-02-07T13:11:13.104 INFO:teuthology.orchestra.run.smithi067.stdout:rgw.foo.a smithi067 *:80 running (77s) 53s ago 77s 74.7M - 19.0.0-1252-g44458cce 79957b372307 d68d5b811d4b 2024-02-07T13:11:13.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:12 smithi159 ceph-mon[27461]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:13.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:12 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:11:13.368 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:12 smithi191 ceph-mon[27623]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:13.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:12 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:11:13.847 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-07T13:11:14.102 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:11:14.114 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:13 smithi191 bash[39114]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-07T13:11:14.114 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:13 smithi191 bash[39114]: Writing manifest to image destination 2024-02-07T13:11:14.369 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 podman[39114]: 2024-02-07 13:11:14.05347509 +0000 UTC m=+2.369092962 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-07T13:11:14.369 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 podman[39114]: 2024-02-07 13:11:14.144218283 +0000 UTC m=+2.459836188 container create 6ca43b66939e74a145ee2ffdfa8eab204acdd532f60989bba944e276a1969799 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-07T13:11:14.698 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 podman[39114]: 2024-02-07 13:11:14.388111778 +0000 UTC m=+2.703729642 container init 6ca43b66939e74a145ee2ffdfa8eab204acdd532f60989bba944e276a1969799 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-07T13:11:14.698 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 podman[39114]: 2024-02-07 13:11:14.39432312 +0000 UTC m=+2.709940984 container start 6ca43b66939e74a145ee2ffdfa8eab204acdd532f60989bba944e276a1969799 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-07T13:11:14.698 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.397Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-07T13:11:14.698 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.397Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-07T13:11:14.698 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.398Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-07T13:11:14.698 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.398Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-07T13:11:14.698 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.398Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-07T13:11:14.698 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.398Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-07T13:11:14.698 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.398Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-07T13:11:14.698 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.398Z caller=node_exporter.go:117 level=info collector=arp 2024-02-07T13:11:14.699 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.398Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-07T13:11:14.699 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.398Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-07T13:11:14.699 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.398Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-07T13:11:14.699 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.398Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-07T13:11:14.699 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.398Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-07T13:11:14.699 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.398Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-07T13:11:14.699 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-07T13:11:14.699 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-07T13:11:14.700 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=edac 2024-02-07T13:11:14.700 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-07T13:11:14.700 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-07T13:11:14.700 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-07T13:11:14.700 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-07T13:11:14.700 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-07T13:11:14.700 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-07T13:11:14.700 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-07T13:11:14.700 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-07T13:11:14.700 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-07T13:11:14.700 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-07T13:11:14.700 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-07T13:11:14.700 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-07T13:11:14.701 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-07T13:11:14.701 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-07T13:11:14.701 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-07T13:11:14.701 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-07T13:11:14.701 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=os 2024-02-07T13:11:14.701 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-07T13:11:14.701 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-07T13:11:14.701 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-07T13:11:14.701 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-07T13:11:14.701 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-07T13:11:14.701 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-07T13:11:14.701 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-07T13:11:14.701 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=stat 2024-02-07T13:11:14.701 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-07T13:11:14.702 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-07T13:11:14.702 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-07T13:11:14.702 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=time 2024-02-07T13:11:14.702 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-07T13:11:14.702 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=uname 2024-02-07T13:11:14.702 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-07T13:11:14.702 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-07T13:11:14.702 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-07T13:11:14.702 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-07T13:11:14.702 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c[39165]: ts=2024-02-07T13:11:14.399Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-07T13:11:14.702 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 bash[39114]: 6ca43b66939e74a145ee2ffdfa8eab204acdd532f60989bba944e276a1969799 2024-02-07T13:11:14.702 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:11:14 smithi191 systemd[1]: Started Ceph node-exporter.c for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:11:14.968 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-mon[27623]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:14.968 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-mon[27623]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:11:14.968 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:14.968 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:14.968 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:14.968 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:14.968 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:14.968 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:11:14.968 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi191", "caps": []}]: dispatch 2024-02-07T13:11:14.968 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi159", "caps": []}]: dispatch 2024-02-07T13:11:14.969 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:14 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi067", "caps": []}]: dispatch 2024-02-07T13:11:15.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:14 smithi159 ceph-mon[27461]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:15.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:14 smithi159 ceph-mon[27461]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:11:15.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:14 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:15.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:14 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:15.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:14 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:15.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:14 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:15.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:14 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:15.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:14 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:11:15.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:14 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi191", "caps": []}]: dispatch 2024-02-07T13:11:15.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:14 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi159", "caps": []}]: dispatch 2024-02-07T13:11:15.194 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:14 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi067", "caps": []}]: dispatch 2024-02-07T13:11:15.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:14 smithi067 ceph-mon[25698]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:15.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:14 smithi067 ceph-mon[25698]: from='client.14559 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:11:15.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:14 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:15.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:14 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:15.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:14 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:15.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:14 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:15.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:14 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:15.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:14 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-07T13:11:15.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:14 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi191", "caps": []}]: dispatch 2024-02-07T13:11:15.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:14 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi159", "caps": []}]: dispatch 2024-02-07T13:11:15.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:14 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi067", "caps": []}]: dispatch 2024-02-07T13:11:16.354 INFO:teuthology.orchestra.run.smithi067.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-07T13:11:16.354 INFO:teuthology.orchestra.run.smithi067.stdout:agent 2/3 66s ago 49s * 2024-02-07T13:11:16.354 INFO:teuthology.orchestra.run.smithi067.stdout:alertmanager ?:9093,9094 0/1 - 74s smithi067=a;count:1 2024-02-07T13:11:16.354 INFO:teuthology.orchestra.run.smithi067.stdout:grafana ?:3000 0/1 - 71s smithi159=a;count:1 2024-02-07T13:11:16.354 INFO:teuthology.orchestra.run.smithi067.stdout:mgr 3/3 68s ago 2m smithi067=y;smithi159=x;smithi191=z;count:3 2024-02-07T13:11:16.354 INFO:teuthology.orchestra.run.smithi067.stdout:mon 3/3 68s ago 3m smithi067:172.21.15.67=a;smithi159:172.21.15.159=b;smithi191:172.21.15.191=c;count:3 2024-02-07T13:11:16.354 INFO:teuthology.orchestra.run.smithi067.stdout:node-exporter ?:9100 0/3 - 77s smithi067=a;smithi159=b;smithi191=c;count:3 2024-02-07T13:11:16.354 INFO:teuthology.orchestra.run.smithi067.stdout:osd 3 68s ago - 2024-02-07T13:11:16.354 INFO:teuthology.orchestra.run.smithi067.stdout:prometheus ?:9095 0/1 - 80s smithi159=a;count:1 2024-02-07T13:11:16.354 INFO:teuthology.orchestra.run.smithi067.stdout:rgw.foo.a ?:80 1/1 57s ago 80s smithi067=foo.a;count:1 2024-02-07T13:11:17.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:16 smithi067 ceph-mon[25698]: pgmap v27: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:17.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:16 smithi067 ceph-mon[25698]: Reconfiguring daemon agent.smithi159 on smithi159 2024-02-07T13:11:17.025 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:16 smithi067 ceph-mon[25698]: Reconfiguring daemon agent.smithi191 on smithi191 2024-02-07T13:11:17.026 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:16 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:17.026 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:16 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:17.122 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:16 smithi159 ceph-mon[27461]: pgmap v27: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:17.123 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:16 smithi159 ceph-mon[27461]: Reconfiguring daemon agent.smithi159 on smithi159 2024-02-07T13:11:17.123 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:16 smithi159 ceph-mon[27461]: Reconfiguring daemon agent.smithi191 on smithi191 2024-02-07T13:11:17.123 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:16 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:17.123 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:16 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:17.163 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:16 smithi191 ceph-mon[27623]: pgmap v27: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:17.163 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:16 smithi191 ceph-mon[27623]: Reconfiguring daemon agent.smithi159 on smithi159 2024-02-07T13:11:17.163 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:16 smithi191 ceph-mon[27623]: Reconfiguring daemon agent.smithi191 on smithi191 2024-02-07T13:11:17.163 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:16 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:17.163 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:16 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:18.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:17 smithi067 ceph-mon[25698]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:11:18.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:17 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:18.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:17 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:18.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:17 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:18.179 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:17 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:18.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:17 smithi159 ceph-mon[27461]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:11:18.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:17 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:18.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:17 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:18.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:17 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:18.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:17 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:18.343 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:17 smithi191 ceph-mon[27623]: from='client.14565 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:11:18.343 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:17 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:18.343 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:17 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:18.343 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:17 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:18.343 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:17 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:18.720 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-07T13:11:18.976 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:18 smithi067 ceph-mon[25698]: pgmap v28: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:19.169 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:18 smithi159 ceph-mon[27461]: pgmap v28: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:19.347 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:18 smithi191 ceph-mon[27623]: pgmap v28: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:20.123 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:11:21.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:20 smithi067 ceph-mon[25698]: pgmap v29: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:21.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:20 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:21.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:20 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:21.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:20 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:21.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:20 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:21.056 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:20 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:21.061 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:20 smithi159 ceph-mon[27461]: pgmap v29: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:21.062 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:20 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:21.062 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:20 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:21.062 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:20 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:21.062 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:20 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:21.062 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:20 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:21.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:20 smithi191 ceph-mon[27623]: pgmap v29: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:21.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:20 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:21.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:20 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:21.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:20 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:21.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:20 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:21.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:20 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:21.506 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:21 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: ::ffff:172.21.15.159 - - [07/Feb/2024:13:11:21] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-07T13:11:22.998 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:22 smithi067 ceph-mon[25698]: pgmap v30: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:22.998 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:22 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:22.998 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:22 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:23.118 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:22 smithi191 ceph-mon[27623]: pgmap v30: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:23.118 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:22 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:23.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:22 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:23.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:22 smithi159 ceph-mon[27461]: pgmap v30: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:23.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:22 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:23.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:22 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:23.551 INFO:teuthology.orchestra.run.smithi067.stdout:HOST ADDR LABELS STATUS 2024-02-07T13:11:23.551 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067 172.21.15.67 2024-02-07T13:11:23.551 INFO:teuthology.orchestra.run.smithi067.stdout:smithi159 172.21.15.159 2024-02-07T13:11:23.551 INFO:teuthology.orchestra.run.smithi067.stdout:smithi191 172.21.15.191 2024-02-07T13:11:23.551 INFO:teuthology.orchestra.run.smithi067.stdout:3 hosts in cluster 2024-02-07T13:11:24.118 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:24 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:24.118 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:24 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:24.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:24 smithi191 ceph-mon[27623]: pgmap v31: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:24.119 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:24 smithi191 ceph-mon[27623]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:11:24.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:24 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:24.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:24 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:24.305 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:24 smithi067 ceph-mon[25698]: pgmap v31: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:24.306 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:24 smithi067 ceph-mon[25698]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:11:24.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:24 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:24.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:24 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:24.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:24 smithi159 ceph-mon[27461]: pgmap v31: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:24.443 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:24 smithi159 ceph-mon[27461]: from='client.14571 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:11:24.783 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-07T13:11:25.027 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:11:25.613 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:25 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:25.613 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:25 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:25.613 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:25 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:25.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:25 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:25.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:25 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:25.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:25 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:25.867 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:25 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:25.867 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:25 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:25.868 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:25 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:26.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:26 smithi067 ceph-mon[25698]: pgmap v32: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:26.618 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:26 smithi191 ceph-mon[27623]: pgmap v32: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:26.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:26 smithi159 ceph-mon[27461]: pgmap v32: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:27.027 INFO:teuthology.orchestra.run.smithi067.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-07T13:11:27.027 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53440042400BGN 372G No 2s ago Has a FileSystem, LVM detected 2024-02-07T13:11:27.027 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067 /dev/nvme1n1 ssd Linux_1ed150da14ac69614e34 89.4G Yes 2s ago 2024-02-07T13:11:27.027 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067 /dev/nvme2n1 ssd Linux_b4027e7d11be2526586f 89.4G Yes 2s ago 2024-02-07T13:11:27.027 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067 /dev/nvme3n1 ssd Linux_f627b8f5491ef3b8ef19 89.4G Yes 2s ago 2024-02-07T13:11:27.027 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067 /dev/nvme4n1 ssd Linux_4f2c04494dc72fca7467 89.4G No 2s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-07T13:11:27.027 INFO:teuthology.orchestra.run.smithi067.stdout:smithi067 /dev/sda hdd ATA_ST1000NM0033-9ZM173_Z1W4W31M 931G No 2s ago Has partitions 2024-02-07T13:11:27.027 INFO:teuthology.orchestra.run.smithi067.stdout:smithi159 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300GH400BGN 372G No 79s ago Has a FileSystem, LVM detected 2024-02-07T13:11:27.027 INFO:teuthology.orchestra.run.smithi067.stdout:smithi159 /dev/nvme1n1 ssd Linux_c46879dfccccf212f527 89.4G Yes 79s ago 2024-02-07T13:11:27.027 INFO:teuthology.orchestra.run.smithi067.stdout:smithi159 /dev/nvme2n1 ssd Linux_457c5bb7b695c5a9cff9 89.4G Yes 79s ago 2024-02-07T13:11:27.027 INFO:teuthology.orchestra.run.smithi067.stdout:smithi159 /dev/nvme3n1 ssd Linux_6d79fe0dd1759adaa4db 89.4G Yes 79s ago 2024-02-07T13:11:27.027 INFO:teuthology.orchestra.run.smithi067.stdout:smithi159 /dev/nvme4n1 ssd Linux_0906f27ba2f040807628 89.4G No 79s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-07T13:11:27.027 INFO:teuthology.orchestra.run.smithi067.stdout:smithi159 /dev/sda hdd ATA_ST1000NM0033-9ZM173_Z1W5XLPF 931G No 79s ago Has partitions 2024-02-07T13:11:27.028 INFO:teuthology.orchestra.run.smithi067.stdout:smithi191 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303008M400BGN 372G No 0s ago Has a FileSystem, LVM detected 2024-02-07T13:11:27.028 INFO:teuthology.orchestra.run.smithi067.stdout:smithi191 /dev/nvme1n1 ssd Linux_468ca342ac84082eabee 89.4G Yes 0s ago 2024-02-07T13:11:27.028 INFO:teuthology.orchestra.run.smithi067.stdout:smithi191 /dev/nvme2n1 ssd Linux_aa3cdf8222426e25d7a2 89.4G Yes 0s ago 2024-02-07T13:11:27.028 INFO:teuthology.orchestra.run.smithi067.stdout:smithi191 /dev/nvme3n1 ssd Linux_98bc860a0cce584729f6 89.4G Yes 0s ago 2024-02-07T13:11:27.028 INFO:teuthology.orchestra.run.smithi067.stdout:smithi191 /dev/nvme4n1 ssd Linux_8ed4b22ce8533619e02b 89.4G No 0s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-07T13:11:27.652 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:27 smithi067 ceph-mon[25698]: Detected new or changed devices on smithi191 2024-02-07T13:11:27.652 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:27.652 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:27.652 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:27 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:11:27.679 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- bash -c 'ceph orch ls --format yaml' 2024-02-07T13:11:27.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:27 smithi159 ceph-mon[27461]: Detected new or changed devices on smithi191 2024-02-07T13:11:27.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:27.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:27.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:27 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:11:27.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:27 smithi191 ceph-mon[27623]: Detected new or changed devices on smithi191 2024-02-07T13:11:27.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:27.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:27.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:27 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-07T13:11:27.920 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:11:28.555 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:28 smithi067 ceph-mon[25698]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:11:28.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:28 smithi067 ceph-mon[25698]: pgmap v33: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:28.790 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:28 smithi159 ceph-mon[27461]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:11:28.791 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:28 smithi159 ceph-mon[27461]: pgmap v33: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:28.868 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:28 smithi191 ceph-mon[27623]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:11:28.868 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:28 smithi191 ceph-mon[27623]: pgmap v33: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:29.859 INFO:teuthology.orchestra.run.smithi067.stdout:service_type: agent 2024-02-07T13:11:29.859 INFO:teuthology.orchestra.run.smithi067.stdout:service_name: agent 2024-02-07T13:11:29.860 INFO:teuthology.orchestra.run.smithi067.stdout:placement: 2024-02-07T13:11:29.860 INFO:teuthology.orchestra.run.smithi067.stdout: host_pattern: '*' 2024-02-07T13:11:29.860 INFO:teuthology.orchestra.run.smithi067.stdout:status: 2024-02-07T13:11:29.860 INFO:teuthology.orchestra.run.smithi067.stdout: created: '2024-02-07T13:10:27.109000Z' 2024-02-07T13:11:29.860 INFO:teuthology.orchestra.run.smithi067.stdout: last_refresh: '2024-02-07T13:11:21.952091Z' 2024-02-07T13:11:29.860 INFO:teuthology.orchestra.run.smithi067.stdout: running: 2 2024-02-07T13:11:29.860 INFO:teuthology.orchestra.run.smithi067.stdout: size: 3 2024-02-07T13:11:29.860 INFO:teuthology.orchestra.run.smithi067.stdout:events: 2024-02-07T13:11:29.860 INFO:teuthology.orchestra.run.smithi067.stdout:- 2024-02-07T13:10:28.658470Z service:agent [INFO] "service was created" 2024-02-07T13:11:29.860 INFO:teuthology.orchestra.run.smithi067.stdout:--- 2024-02-07T13:11:29.860 INFO:teuthology.orchestra.run.smithi067.stdout:service_type: alertmanager 2024-02-07T13:11:29.860 INFO:teuthology.orchestra.run.smithi067.stdout:service_name: alertmanager 2024-02-07T13:11:29.860 INFO:teuthology.orchestra.run.smithi067.stdout:placement: 2024-02-07T13:11:29.860 INFO:teuthology.orchestra.run.smithi067.stdout: count: 1 2024-02-07T13:11:29.861 INFO:teuthology.orchestra.run.smithi067.stdout: hosts: 2024-02-07T13:11:29.861 INFO:teuthology.orchestra.run.smithi067.stdout: - smithi067=a 2024-02-07T13:11:29.861 INFO:teuthology.orchestra.run.smithi067.stdout:status: 2024-02-07T13:11:29.861 INFO:teuthology.orchestra.run.smithi067.stdout: created: '2024-02-07T13:10:01.479524Z' 2024-02-07T13:11:29.861 INFO:teuthology.orchestra.run.smithi067.stdout: last_refresh: '2024-02-07T13:11:24.459466Z' 2024-02-07T13:11:29.861 INFO:teuthology.orchestra.run.smithi067.stdout: ports: 2024-02-07T13:11:29.861 INFO:teuthology.orchestra.run.smithi067.stdout: - 9093 2024-02-07T13:11:29.861 INFO:teuthology.orchestra.run.smithi067.stdout: - 9094 2024-02-07T13:11:29.861 INFO:teuthology.orchestra.run.smithi067.stdout: running: 1 2024-02-07T13:11:29.861 INFO:teuthology.orchestra.run.smithi067.stdout: size: 1 2024-02-07T13:11:29.861 INFO:teuthology.orchestra.run.smithi067.stdout:events: 2024-02-07T13:11:29.861 INFO:teuthology.orchestra.run.smithi067.stdout:- 2024-02-07T13:10:44.115615Z service:alertmanager [INFO] "service was created" 2024-02-07T13:11:29.861 INFO:teuthology.orchestra.run.smithi067.stdout:--- 2024-02-07T13:11:29.861 INFO:teuthology.orchestra.run.smithi067.stdout:service_type: grafana 2024-02-07T13:11:29.861 INFO:teuthology.orchestra.run.smithi067.stdout:service_name: grafana 2024-02-07T13:11:29.861 INFO:teuthology.orchestra.run.smithi067.stdout:placement: 2024-02-07T13:11:29.862 INFO:teuthology.orchestra.run.smithi067.stdout: count: 1 2024-02-07T13:11:29.862 INFO:teuthology.orchestra.run.smithi067.stdout: hosts: 2024-02-07T13:11:29.862 INFO:teuthology.orchestra.run.smithi067.stdout: - smithi159=a 2024-02-07T13:11:29.862 INFO:teuthology.orchestra.run.smithi067.stdout:spec: 2024-02-07T13:11:29.862 INFO:teuthology.orchestra.run.smithi067.stdout: anonymous_access: true 2024-02-07T13:11:29.862 INFO:teuthology.orchestra.run.smithi067.stdout: protocol: https 2024-02-07T13:11:29.862 INFO:teuthology.orchestra.run.smithi067.stdout:status: 2024-02-07T13:11:29.862 INFO:teuthology.orchestra.run.smithi067.stdout: created: '2024-02-07T13:10:04.444331Z' 2024-02-07T13:11:29.862 INFO:teuthology.orchestra.run.smithi067.stdout: ports: 2024-02-07T13:11:29.862 INFO:teuthology.orchestra.run.smithi067.stdout: - 3000 2024-02-07T13:11:29.862 INFO:teuthology.orchestra.run.smithi067.stdout: running: 0 2024-02-07T13:11:29.862 INFO:teuthology.orchestra.run.smithi067.stdout: size: 1 2024-02-07T13:11:29.862 INFO:teuthology.orchestra.run.smithi067.stdout:events: 2024-02-07T13:11:29.862 INFO:teuthology.orchestra.run.smithi067.stdout:- 2024-02-07T13:11:00.090496Z service:grafana [INFO] "service was created" 2024-02-07T13:11:29.862 INFO:teuthology.orchestra.run.smithi067.stdout:--- 2024-02-07T13:11:29.862 INFO:teuthology.orchestra.run.smithi067.stdout:service_type: mgr 2024-02-07T13:11:29.863 INFO:teuthology.orchestra.run.smithi067.stdout:service_name: mgr 2024-02-07T13:11:29.863 INFO:teuthology.orchestra.run.smithi067.stdout:placement: 2024-02-07T13:11:29.863 INFO:teuthology.orchestra.run.smithi067.stdout: count: 3 2024-02-07T13:11:29.863 INFO:teuthology.orchestra.run.smithi067.stdout: hosts: 2024-02-07T13:11:29.863 INFO:teuthology.orchestra.run.smithi067.stdout: - smithi067=y 2024-02-07T13:11:29.863 INFO:teuthology.orchestra.run.smithi067.stdout: - smithi159=x 2024-02-07T13:11:29.863 INFO:teuthology.orchestra.run.smithi067.stdout: - smithi191=z 2024-02-07T13:11:29.863 INFO:teuthology.orchestra.run.smithi067.stdout:status: 2024-02-07T13:11:29.863 INFO:teuthology.orchestra.run.smithi067.stdout: created: '2024-02-07T13:08:19.090757Z' 2024-02-07T13:11:29.863 INFO:teuthology.orchestra.run.smithi067.stdout: last_refresh: '2024-02-07T13:10:07.881494Z' 2024-02-07T13:11:29.863 INFO:teuthology.orchestra.run.smithi067.stdout: running: 3 2024-02-07T13:11:29.863 INFO:teuthology.orchestra.run.smithi067.stdout: size: 3 2024-02-07T13:11:29.863 INFO:teuthology.orchestra.run.smithi067.stdout:--- 2024-02-07T13:11:29.863 INFO:teuthology.orchestra.run.smithi067.stdout:service_type: mon 2024-02-07T13:11:29.863 INFO:teuthology.orchestra.run.smithi067.stdout:service_name: mon 2024-02-07T13:11:29.864 INFO:teuthology.orchestra.run.smithi067.stdout:placement: 2024-02-07T13:11:29.864 INFO:teuthology.orchestra.run.smithi067.stdout: count: 3 2024-02-07T13:11:29.864 INFO:teuthology.orchestra.run.smithi067.stdout: hosts: 2024-02-07T13:11:29.864 INFO:teuthology.orchestra.run.smithi067.stdout: - smithi067:172.21.15.67=a 2024-02-07T13:11:29.864 INFO:teuthology.orchestra.run.smithi067.stdout: - smithi159:172.21.15.159=b 2024-02-07T13:11:29.864 INFO:teuthology.orchestra.run.smithi067.stdout: - smithi191:172.21.15.191=c 2024-02-07T13:11:29.864 INFO:teuthology.orchestra.run.smithi067.stdout:status: 2024-02-07T13:11:29.864 INFO:teuthology.orchestra.run.smithi067.stdout: created: '2024-02-07T13:07:52.231239Z' 2024-02-07T13:11:29.864 INFO:teuthology.orchestra.run.smithi067.stdout: last_refresh: '2024-02-07T13:10:07.881404Z' 2024-02-07T13:11:29.864 INFO:teuthology.orchestra.run.smithi067.stdout: running: 3 2024-02-07T13:11:29.864 INFO:teuthology.orchestra.run.smithi067.stdout: size: 3 2024-02-07T13:11:29.864 INFO:teuthology.orchestra.run.smithi067.stdout:--- 2024-02-07T13:11:29.864 INFO:teuthology.orchestra.run.smithi067.stdout:service_type: node-exporter 2024-02-07T13:11:29.864 INFO:teuthology.orchestra.run.smithi067.stdout:service_name: node-exporter 2024-02-07T13:11:29.864 INFO:teuthology.orchestra.run.smithi067.stdout:placement: 2024-02-07T13:11:29.864 INFO:teuthology.orchestra.run.smithi067.stdout: count: 3 2024-02-07T13:11:29.865 INFO:teuthology.orchestra.run.smithi067.stdout: hosts: 2024-02-07T13:11:29.865 INFO:teuthology.orchestra.run.smithi067.stdout: - smithi067=a 2024-02-07T13:11:29.865 INFO:teuthology.orchestra.run.smithi067.stdout: - smithi159=b 2024-02-07T13:11:29.865 INFO:teuthology.orchestra.run.smithi067.stdout: - smithi191=c 2024-02-07T13:11:29.865 INFO:teuthology.orchestra.run.smithi067.stdout:status: 2024-02-07T13:11:29.865 INFO:teuthology.orchestra.run.smithi067.stdout: created: '2024-02-07T13:09:58.519603Z' 2024-02-07T13:11:29.865 INFO:teuthology.orchestra.run.smithi067.stdout: last_refresh: '2024-02-07T13:11:21.952480Z' 2024-02-07T13:11:29.865 INFO:teuthology.orchestra.run.smithi067.stdout: ports: 2024-02-07T13:11:29.865 INFO:teuthology.orchestra.run.smithi067.stdout: - 9100 2024-02-07T13:11:29.865 INFO:teuthology.orchestra.run.smithi067.stdout: running: 2 2024-02-07T13:11:29.865 INFO:teuthology.orchestra.run.smithi067.stdout: size: 3 2024-02-07T13:11:29.865 INFO:teuthology.orchestra.run.smithi067.stdout:events: 2024-02-07T13:11:29.865 INFO:teuthology.orchestra.run.smithi067.stdout:- 2024-02-07T13:11:14.575277Z service:node-exporter [INFO] "service was created" 2024-02-07T13:11:29.865 INFO:teuthology.orchestra.run.smithi067.stdout:--- 2024-02-07T13:11:29.865 INFO:teuthology.orchestra.run.smithi067.stdout:service_type: osd 2024-02-07T13:11:29.866 INFO:teuthology.orchestra.run.smithi067.stdout:service_name: osd 2024-02-07T13:11:29.866 INFO:teuthology.orchestra.run.smithi067.stdout:unmanaged: true 2024-02-07T13:11:29.866 INFO:teuthology.orchestra.run.smithi067.stdout:spec: 2024-02-07T13:11:29.866 INFO:teuthology.orchestra.run.smithi067.stdout: filter_logic: AND 2024-02-07T13:11:29.866 INFO:teuthology.orchestra.run.smithi067.stdout: objectstore: bluestore 2024-02-07T13:11:29.866 INFO:teuthology.orchestra.run.smithi067.stdout:status: 2024-02-07T13:11:29.866 INFO:teuthology.orchestra.run.smithi067.stdout: container_image_id: 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f 2024-02-07T13:11:29.866 INFO:teuthology.orchestra.run.smithi067.stdout: container_image_name: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T13:11:29.866 INFO:teuthology.orchestra.run.smithi067.stdout: last_refresh: '2024-02-07T13:10:07.881579Z' 2024-02-07T13:11:29.866 INFO:teuthology.orchestra.run.smithi067.stdout: running: 3 2024-02-07T13:11:29.866 INFO:teuthology.orchestra.run.smithi067.stdout: size: 3 2024-02-07T13:11:29.866 INFO:teuthology.orchestra.run.smithi067.stdout:--- 2024-02-07T13:11:29.866 INFO:teuthology.orchestra.run.smithi067.stdout:service_type: prometheus 2024-02-07T13:11:29.866 INFO:teuthology.orchestra.run.smithi067.stdout:service_name: prometheus 2024-02-07T13:11:29.866 INFO:teuthology.orchestra.run.smithi067.stdout:placement: 2024-02-07T13:11:29.866 INFO:teuthology.orchestra.run.smithi067.stdout: count: 1 2024-02-07T13:11:29.867 INFO:teuthology.orchestra.run.smithi067.stdout: hosts: 2024-02-07T13:11:29.867 INFO:teuthology.orchestra.run.smithi067.stdout: - smithi159=a 2024-02-07T13:11:29.867 INFO:teuthology.orchestra.run.smithi067.stdout:status: 2024-02-07T13:11:29.867 INFO:teuthology.orchestra.run.smithi067.stdout: created: '2024-02-07T13:09:55.891723Z' 2024-02-07T13:11:29.867 INFO:teuthology.orchestra.run.smithi067.stdout: ports: 2024-02-07T13:11:29.867 INFO:teuthology.orchestra.run.smithi067.stdout: - 9095 2024-02-07T13:11:29.867 INFO:teuthology.orchestra.run.smithi067.stdout: running: 0 2024-02-07T13:11:29.867 INFO:teuthology.orchestra.run.smithi067.stdout: size: 1 2024-02-07T13:11:29.867 INFO:teuthology.orchestra.run.smithi067.stdout:events: 2024-02-07T13:11:29.867 INFO:teuthology.orchestra.run.smithi067.stdout:- 2024-02-07T13:11:14.583547Z service:prometheus [INFO] "service was created" 2024-02-07T13:11:29.867 INFO:teuthology.orchestra.run.smithi067.stdout:--- 2024-02-07T13:11:29.867 INFO:teuthology.orchestra.run.smithi067.stdout:service_type: rgw 2024-02-07T13:11:29.867 INFO:teuthology.orchestra.run.smithi067.stdout:service_id: foo.a 2024-02-07T13:11:29.867 INFO:teuthology.orchestra.run.smithi067.stdout:service_name: rgw.foo.a 2024-02-07T13:11:29.867 INFO:teuthology.orchestra.run.smithi067.stdout:placement: 2024-02-07T13:11:29.867 INFO:teuthology.orchestra.run.smithi067.stdout: count: 1 2024-02-07T13:11:29.868 INFO:teuthology.orchestra.run.smithi067.stdout: hosts: 2024-02-07T13:11:29.868 INFO:teuthology.orchestra.run.smithi067.stdout: - smithi067=foo.a 2024-02-07T13:11:29.868 INFO:teuthology.orchestra.run.smithi067.stdout:status: 2024-02-07T13:11:29.868 INFO:teuthology.orchestra.run.smithi067.stdout: created: '2024-02-07T13:09:55.894300Z' 2024-02-07T13:11:29.868 INFO:teuthology.orchestra.run.smithi067.stdout: last_refresh: '2024-02-07T13:11:24.459392Z' 2024-02-07T13:11:29.868 INFO:teuthology.orchestra.run.smithi067.stdout: ports: 2024-02-07T13:11:29.868 INFO:teuthology.orchestra.run.smithi067.stdout: - 80 2024-02-07T13:11:29.868 INFO:teuthology.orchestra.run.smithi067.stdout: running: 1 2024-02-07T13:11:29.868 INFO:teuthology.orchestra.run.smithi067.stdout: size: 1 2024-02-07T13:11:30.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-02-07T13:11:30.732 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/mon.a/config 2024-02-07T13:11:31.044 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:30 smithi159 ceph-mon[27461]: pgmap v34: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:31.045 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:30 smithi159 ceph-mon[27461]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-07T13:11:31.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:30 smithi067 ceph-mon[25698]: pgmap v34: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:31.185 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:30 smithi067 ceph-mon[25698]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-07T13:11:31.368 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:30 smithi191 ceph-mon[27623]: pgmap v34: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:31.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:30 smithi191 ceph-mon[27623]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-07T13:11:31.555 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:31 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: ::ffff:172.21.15.159 - - [07/Feb/2024:13:11:31] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-07T13:11:32.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:32 smithi067 ceph-mon[25698]: pgmap v35: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:32.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:32 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:32.556 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:32 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:32.618 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:32 smithi191 ceph-mon[27623]: pgmap v35: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:32.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:32 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:32.619 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:32 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:32.664 INFO:teuthology.orchestra.run.smithi067.stdout:osd 3 10s ago - 2024-02-07T13:11:32.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:32 smithi159 ceph-mon[27461]: pgmap v35: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-07T13:11:32.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:32 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:32.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:32 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:33.391 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-07T13:11:33.400 INFO:tasks.cephadm:Teardown begin 2024-02-07T13:11:33.401 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-07T13:11:33.431 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-07T13:11:33.460 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-07T13:11:33.497 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-07T13:11:33.497 DEBUG:teuthology.orchestra.run.smithi067:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-07T13:11:33.513 DEBUG:teuthology.orchestra.run.smithi159:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-07T13:11:33.528 DEBUG:teuthology.orchestra.run.smithi191:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-07T13:11:33.555 INFO:tasks.cephadm:Stopping all daemons... 2024-02-07T13:11:33.555 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-07T13:11:33.555 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mon.a 2024-02-07T13:11:33.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:33 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:33.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:33 smithi159 ceph-mon[27461]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:33.693 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:33 smithi159 ceph-mon[27461]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:11:33.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:33 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:33.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:33 smithi067 ceph-mon[25698]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:33.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:33 smithi067 ceph-mon[25698]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:11:33.806 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:33 smithi067 systemd[1]: Stopping Ceph mon.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:11:33.868 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:33 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:33.868 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:33 smithi191 ceph-mon[27623]: from='mgr.14451 172.21.15.67:0/1469188532' entity='mgr.y' 2024-02-07T13:11:33.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:33 smithi191 ceph-mon[27623]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-07T13:11:34.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:33 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-a[25675]: 2024-02-07T13:11:33.859+0000 7faf02079700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T13:11:34.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:33 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-a[25675]: 2024-02-07T13:11:33.859+0000 7faf02079700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-07T13:11:34.183 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:33 smithi067 podman[52648]: 2024-02-07 13:11:33.93110675 +0000 UTC m=+0.181431793 container died 4aae939770198905808bc85f38a03e2a5d433e80eca80b01a9871444345a6c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc) 2024-02-07T13:11:34.183 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:33 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:11:33] ENGINE Bus STOPPING 2024-02-07T13:11:34.183 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:33 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:11:33] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-07T13:11:34.183 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:33 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:11:33] ENGINE Bus STOPPED 2024-02-07T13:11:34.184 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:33 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:11:33] ENGINE Bus STARTING 2024-02-07T13:11:34.184 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:33 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:11:33] ENGINE Serving on http://:::9283 2024-02-07T13:11:34.184 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:33 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:11:33] ENGINE Bus STARTED 2024-02-07T13:11:34.533 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:34 smithi067 podman[52648]: 2024-02-07 13:11:34.365519858 +0000 UTC m=+0.615844899 container cleanup 4aae939770198905808bc85f38a03e2a5d433e80eca80b01a9871444345a6c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-a, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-02-07T13:11:34.533 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:34 smithi067 bash[52648]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-a 2024-02-07T13:11:34.759 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mon.a.service' 2024-02-07T13:11:34.784 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:34 smithi067 podman[52678]: 2024-02-07 13:11:34.533365591 +0000 UTC m=+0.606801583 container remove 4aae939770198905808bc85f38a03e2a5d433e80eca80b01a9871444345a6c8c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, ceph=True, org.label-schema.build-date=20240131, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-07T13:11:34.784 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:34 smithi067 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-07T13:11:34.784 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:34 smithi067 systemd[1]: Stopped Ceph mon.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:11:34.784 INFO:journalctl@ceph.mon.a.smithi067.stdout:Feb 07 13:11:34 smithi067 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mon.a.service: Consumed 8.726s CPU time. 2024-02-07T13:11:34.822 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:11:34.822 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-07T13:11:34.822 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-02-07T13:11:34.823 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mon.b 2024-02-07T13:11:35.143 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:34 smithi159 systemd[1]: Stopping Ceph mon.b for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:11:35.429 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:35 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-b[27438]: 2024-02-07T13:11:35.140+0000 7f206d926700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T13:11:35.429 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:35 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-b[27438]: 2024-02-07T13:11:35.140+0000 7f206d926700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-02-07T13:11:35.429 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:35 smithi159 podman[41743]: 2024-02-07 13:11:35.294754266 +0000 UTC m=+0.276690754 container died fe28fbc6f0a8e443f7378a579643174e4883d301c69ed5a6a46d4eab051f8cc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-02-07T13:11:35.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:35 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:11:35] ENGINE Bus STOPPING 2024-02-07T13:11:35.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:35 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:11:35] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-07T13:11:35.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:35 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:11:35] ENGINE Bus STOPPED 2024-02-07T13:11:35.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:35 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:11:35] ENGINE Bus STARTING 2024-02-07T13:11:35.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:35 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:11:35] ENGINE Serving on http://:::9283 2024-02-07T13:11:35.556 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:35 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: [07/Feb/2024:13:11:35] ENGINE Bus STARTED 2024-02-07T13:11:36.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:35 smithi159 podman[41743]: 2024-02-07 13:11:35.731151906 +0000 UTC m=+0.713088396 container cleanup fe28fbc6f0a8e443f7378a579643174e4883d301c69ed5a6a46d4eab051f8cc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-b, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_CLEAN=True, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-02-07T13:11:36.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:35 smithi159 bash[41743]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-b 2024-02-07T13:11:36.193 INFO:journalctl@ceph.mon.b.smithi159.stdout:Feb 07 13:11:35 smithi159 podman[44488]: 2024-02-07 13:11:35.88212794 +0000 UTC m=+0.590826046 container remove fe28fbc6f0a8e443f7378a579643174e4883d301c69ed5a6a46d4eab051f8cc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc) 2024-02-07T13:11:36.721 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mon.b.service' 2024-02-07T13:11:36.771 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:11:36.771 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-02-07T13:11:36.771 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-02-07T13:11:36.771 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mon.c 2024-02-07T13:11:37.095 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:36 smithi191 systemd[1]: Stopping Ceph mon.c for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:11:37.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:37 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-c[27587]: 2024-02-07T13:11:37.093+0000 7fdc6adfe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T13:11:37.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:37 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-c[27587]: 2024-02-07T13:11:37.093+0000 7fdc6adfe700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-02-07T13:11:37.369 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:37 smithi191 podman[44583]: 2024-02-07 13:11:37.318455185 +0000 UTC m=+0.351553198 container died f1e3ce748477ef97631d64f1d77c519a7b344b08dc256c2d36ba0c4b79ce8d36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131) 2024-02-07T13:11:37.868 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:37 smithi191 podman[44583]: 2024-02-07 13:11:37.666126727 +0000 UTC m=+0.699224741 container cleanup f1e3ce748477ef97631d64f1d77c519a7b344b08dc256c2d36ba0c4b79ce8d36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-07T13:11:37.869 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:37 smithi191 bash[44583]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-c 2024-02-07T13:11:38.237 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mon.c.service' 2024-02-07T13:11:38.244 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:37 smithi191 podman[44602]: 2024-02-07 13:11:37.993171603 +0000 UTC m=+0.676573857 container remove f1e3ce748477ef97631d64f1d77c519a7b344b08dc256c2d36ba0c4b79ce8d36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mon-c, GIT_CLEAN=True, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS) 2024-02-07T13:11:38.244 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:38 smithi191 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mon.c.service: Deactivated successfully. 2024-02-07T13:11:38.244 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:38 smithi191 systemd[1]: Stopped Ceph mon.c for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:11:38.244 INFO:journalctl@ceph.mon.c.smithi191.stdout:Feb 07 13:11:38 smithi191 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mon.c.service: Consumed 4.144s CPU time. 2024-02-07T13:11:38.304 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:11:38.304 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-02-07T13:11:38.304 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-07T13:11:38.304 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mgr.y 2024-02-07T13:11:38.669 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:38 smithi067 systemd[1]: Stopping Ceph mgr.y for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:11:38.943 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:38 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:11:38.667+0000 7ff356d60700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T13:11:38.943 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:38 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y[25955]: 2024-02-07T13:11:38.668+0000 7ff356d60700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-07T13:11:38.943 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:38 smithi067 podman[53018]: 2024-02-07 13:11:38.699289567 +0000 UTC m=+0.150426293 container died c67bb53a16c70efa315f9e094a7c58fbf35e3b7db8a13f79904fb5add555d662 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-07T13:11:39.302 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:39 smithi067 podman[53018]: 2024-02-07 13:11:39.117359559 +0000 UTC m=+0.568496346 container cleanup c67bb53a16c70efa315f9e094a7c58fbf35e3b7db8a13f79904fb5add555d662 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-07T13:11:39.302 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:39 smithi067 bash[53018]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y 2024-02-07T13:11:39.555 INFO:journalctl@ceph.mgr.y.smithi067.stdout:Feb 07 13:11:39 smithi067 podman[53063]: 2024-02-07 13:11:39.302166437 +0000 UTC m=+0.601695443 container remove c67bb53a16c70efa315f9e094a7c58fbf35e3b7db8a13f79904fb5add555d662 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:44458ccedb18241cc2e17f10cdab3d1385399261, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-y, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-02-07T13:11:39.724 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mgr.y.service' 2024-02-07T13:11:39.762 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:11:39.762 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-07T13:11:39.762 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-07T13:11:39.762 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mgr.x 2024-02-07T13:11:40.087 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:11:39 smithi159 systemd[1]: Stopping Ceph mgr.x for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:11:40.397 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:11:40 smithi159 podman[46066]: 2024-02-07 13:11:40.087140965 +0000 UTC m=+0.130257673 container died 86d280d6c1bf01d10da5f5e46f102818f3015620f546856a16f9df9d37e75dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, ceph=True) 2024-02-07T13:11:40.693 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:11:40 smithi159 podman[46066]: 2024-02-07 13:11:40.471338915 +0000 UTC m=+0.514455625 container cleanup 86d280d6c1bf01d10da5f5e46f102818f3015620f546856a16f9df9d37e75dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-02-07T13:11:40.693 INFO:journalctl@ceph.mgr.x.smithi159.stdout:Feb 07 13:11:40 smithi159 bash[46066]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-x 2024-02-07T13:11:40.839 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mgr.x.service' 2024-02-07T13:11:40.875 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:11:40.875 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-07T13:11:40.875 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-02-07T13:11:40.876 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mgr.z 2024-02-07T13:11:41.119 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:11:40 smithi191 systemd[1]: Stopping Ceph mgr.z for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:11:42.119 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:11:41 smithi191 podman[45197]: 2024-02-07 13:11:41.906804815 +0000 UTC m=+0.476054735 container died b66654ac8d280fc9286acde1be31f61ea2bb73edca66cc541ff3be082b8a9f64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-07T13:11:42.499 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:11:42 smithi191 podman[45197]: 2024-02-07 13:11:42.280948179 +0000 UTC m=+0.850198100 container cleanup b66654ac8d280fc9286acde1be31f61ea2bb73edca66cc541ff3be082b8a9f64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS) 2024-02-07T13:11:42.499 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:11:42 smithi191 bash[45197]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z 2024-02-07T13:11:42.739 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mgr.z.service' 2024-02-07T13:11:42.763 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:11:42 smithi191 podman[45228]: 2024-02-07 13:11:42.499757423 +0000 UTC m=+0.813798006 container remove b66654ac8d280fc9286acde1be31f61ea2bb73edca66cc541ff3be082b8a9f64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-mgr-z, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.schema-version=1.0) 2024-02-07T13:11:42.763 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:11:42 smithi191 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-02-07T13:11:42.763 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:11:42 smithi191 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mgr.z.service: Failed with result 'exit-code'. 2024-02-07T13:11:42.764 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:11:42 smithi191 systemd[1]: Stopped Ceph mgr.z for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:11:42.764 INFO:journalctl@ceph.mgr.z.smithi191.stdout:Feb 07 13:11:42 smithi191 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@mgr.z.service: Consumed 16.814s CPU time. 2024-02-07T13:11:42.778 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:11:42.778 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-02-07T13:11:42.778 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-07T13:11:42.778 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@osd.0 2024-02-07T13:11:43.055 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:11:42 smithi067 systemd[1]: Stopping Ceph osd.0 for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:11:43.446 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:11:43 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-0[37869]: 2024-02-07T13:11:43.088+0000 7ff2df505700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T13:11:43.446 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:11:43 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-0[37869]: 2024-02-07T13:11:43.088+0000 7ff2df505700 -1 osd.0 32 *** Got signal Terminated *** 2024-02-07T13:11:43.446 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:11:43 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-0[37869]: 2024-02-07T13:11:43.088+0000 7ff2df505700 -1 osd.0 32 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-07T13:11:48.860 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:11:48 smithi067 podman[53523]: 2024-02-07 13:11:48.60318539 +0000 UTC m=+5.620859006 container died 512fdcca12ba5221479307c61628ed0027bac1ec3cfe82746d85c584394735a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-0, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS) 2024-02-07T13:11:49.305 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:11:49 smithi067 podman[53523]: 2024-02-07 13:11:49.073518909 +0000 UTC m=+6.091192524 container cleanup 512fdcca12ba5221479307c61628ed0027bac1ec3cfe82746d85c584394735a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-07T13:11:49.306 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:11:49 smithi067 bash[53523]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-0 2024-02-07T13:11:49.683 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:11:49 smithi067 podman[53585]: 2024-02-07 13:11:49.353111737 +0000 UTC m=+0.749980273 container remove 512fdcca12ba5221479307c61628ed0027bac1ec3cfe82746d85c584394735a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-0, ceph=True, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-07T13:11:49.978 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:11:49 smithi067 podman[53684]: 2024-02-07 13:11:49.586375696 +0000 UTC m=+0.018019109 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T13:11:49.978 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:11:49 smithi067 podman[53684]: 2024-02-07 13:11:49.692469704 +0000 UTC m=+0.124113109 container create 2a372bcdff7d1c76b9d5eaae7433fc3a2631efa53e266bcdba34c976f6d15914 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.2) 2024-02-07T13:11:49.978 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:11:49 smithi067 podman[53684]: 2024-02-07 13:11:49.89446945 +0000 UTC m=+0.326112855 container init 2a372bcdff7d1c76b9d5eaae7433fc3a2631efa53e266bcdba34c976f6d15914 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True) 2024-02-07T13:11:49.978 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:11:49 smithi067 podman[53684]: 2024-02-07 13:11:49.899914653 +0000 UTC m=+0.331558058 container start 2a372bcdff7d1c76b9d5eaae7433fc3a2631efa53e266bcdba34c976f6d15914 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-07T13:11:50.306 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:11:49 smithi067 podman[53684]: 2024-02-07 13:11:49.977818437 +0000 UTC m=+0.409461854 container attach 2a372bcdff7d1c76b9d5eaae7433fc3a2631efa53e266bcdba34c976f6d15914 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-0-deactivate, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, ceph=True) 2024-02-07T13:11:50.306 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:11:50 smithi067 podman[53684]: 2024-02-07 13:11:50.197027942 +0000 UTC m=+0.628671348 container died 2a372bcdff7d1c76b9d5eaae7433fc3a2631efa53e266bcdba34c976f6d15914 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_CLEAN=True, org.label-schema.build-date=20240131, ceph=True, io.buildah.version=1.33.2) 2024-02-07T13:11:50.748 INFO:journalctl@ceph.osd.0.smithi067.stdout:Feb 07 13:11:50 smithi067 podman[53730]: 2024-02-07 13:11:50.730925397 +0000 UTC m=+0.528565634 container remove 2a372bcdff7d1c76b9d5eaae7433fc3a2631efa53e266bcdba34c976f6d15914 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-0-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux ) 2024-02-07T13:11:50.753 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@osd.0.service' 2024-02-07T13:11:50.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:11:50.797 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-07T13:11:50.797 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-07T13:11:50.797 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@osd.1 2024-02-07T13:11:51.104 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:11:50 smithi159 systemd[1]: Stopping Ceph osd.1 for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:11:51.443 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:11:51 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-1[33092]: 2024-02-07T13:11:51.103+0000 7fd537462700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T13:11:51.443 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:11:51 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-1[33092]: 2024-02-07T13:11:51.103+0000 7fd537462700 -1 osd.1 32 *** Got signal Terminated *** 2024-02-07T13:11:51.443 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:11:51 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-1[33092]: 2024-02-07T13:11:51.103+0000 7fd537462700 -1 osd.1 32 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-07T13:11:56.693 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:11:56 smithi159 podman[46916]: 2024-02-07 13:11:56.442212968 +0000 UTC m=+5.439419917 container died d162295b5a5dc30da772976399773a77f5d6d03f877cc76d0a21781b14f807a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-1, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-02-07T13:11:57.001 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:11:56 smithi159 podman[46916]: 2024-02-07 13:11:56.833318822 +0000 UTC m=+5.830525772 container cleanup d162295b5a5dc30da772976399773a77f5d6d03f877cc76d0a21781b14f807a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, ceph=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-07T13:11:57.001 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:11:56 smithi159 bash[46916]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-1 2024-02-07T13:11:57.434 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:11:57 smithi159 podman[46938]: 2024-02-07 13:11:57.000698975 +0000 UTC m=+0.554154418 container remove d162295b5a5dc30da772976399773a77f5d6d03f877cc76d0a21781b14f807a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-1, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-07T13:11:57.693 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:11:57 smithi159 podman[47038]: 2024-02-07 13:11:57.434813557 +0000 UTC m=+0.113738625 container create d5c2b79db41e95dbc837ca38a6de9f1bd23d15d06c34a1bfe82d4016e0e9ecdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-1-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-07T13:11:57.693 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:11:57 smithi159 podman[47038]: 2024-02-07 13:11:57.340411915 +0000 UTC m=+0.019336984 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T13:11:57.693 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:11:57 smithi159 podman[47038]: 2024-02-07 13:11:57.629153474 +0000 UTC m=+0.308078543 container init d5c2b79db41e95dbc837ca38a6de9f1bd23d15d06c34a1bfe82d4016e0e9ecdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-1-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-02-07T13:11:57.693 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:11:57 smithi159 podman[47038]: 2024-02-07 13:11:57.634779731 +0000 UTC m=+0.313704800 container start d5c2b79db41e95dbc837ca38a6de9f1bd23d15d06c34a1bfe82d4016e0e9ecdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-07T13:11:58.193 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:11:57 smithi159 podman[47038]: 2024-02-07 13:11:57.729177899 +0000 UTC m=+0.408102970 container attach d5c2b79db41e95dbc837ca38a6de9f1bd23d15d06c34a1bfe82d4016e0e9ecdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-07T13:11:58.193 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:11:57 smithi159 podman[47038]: 2024-02-07 13:11:57.933773137 +0000 UTC m=+0.612698216 container died d5c2b79db41e95dbc837ca38a6de9f1bd23d15d06c34a1bfe82d4016e0e9ecdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-07T13:11:58.658 INFO:journalctl@ceph.osd.1.smithi159.stdout:Feb 07 13:11:58 smithi159 podman[47084]: 2024-02-07 13:11:58.65644091 +0000 UTC m=+0.713834699 container remove d5c2b79db41e95dbc837ca38a6de9f1bd23d15d06c34a1bfe82d4016e0e9ecdb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-07T13:11:58.682 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@osd.1.service' 2024-02-07T13:11:58.730 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:11:58.730 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-07T13:11:58.730 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-07T13:11:58.730 DEBUG:teuthology.orchestra.run.smithi191:> sudo systemctl stop ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@osd.2 2024-02-07T13:11:59.022 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:11:58 smithi191 systemd[1]: Stopping Ceph osd.2 for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:12:00.119 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:11:59 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-2[34299]: 2024-02-07T13:11:59.860+0000 7f9b4d616700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T13:12:00.119 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:11:59 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-2[34299]: 2024-02-07T13:11:59.860+0000 7f9b4d616700 -1 osd.2 32 *** Got signal Terminated *** 2024-02-07T13:12:00.119 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:11:59 smithi191 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-2[34299]: 2024-02-07T13:11:59.860+0000 7f9b4d616700 -1 osd.2 32 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-07T13:12:05.497 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:12:05 smithi191 podman[45682]: 2024-02-07 13:12:05.225506336 +0000 UTC m=+5.777965097 container died d351f9fcee14e9d142125bf1aeffaf9414ae889d8fdf345c06bb5a19495a4c19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0) 2024-02-07T13:12:05.854 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:12:05 smithi191 podman[45682]: 2024-02-07 13:12:05.570006966 +0000 UTC m=+6.122465725 container cleanup d351f9fcee14e9d142125bf1aeffaf9414ae889d8fdf345c06bb5a19495a4c19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_CLEAN=True) 2024-02-07T13:12:05.854 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:12:05 smithi191 bash[45682]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-2 2024-02-07T13:12:06.118 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:12:05 smithi191 podman[45956]: 2024-02-07 13:12:05.854170641 +0000 UTC m=+0.630753852 container remove d351f9fcee14e9d142125bf1aeffaf9414ae889d8fdf345c06bb5a19495a4c19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_BRANCH=HEAD) 2024-02-07T13:12:06.449 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:12:06 smithi191 podman[46051]: 2024-02-07 13:12:06.081728331 +0000 UTC m=+0.017795137 image pull 79957b3723074dc4d7b43e7345862a464daec6a6b69febcfe25c399b70e0301f quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe 2024-02-07T13:12:06.449 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:12:06 smithi191 podman[46051]: 2024-02-07 13:12:06.205176933 +0000 UTC m=+0.141243734 container create 156c69648757d77f05fb9292fcf1548e10ce632f81d18c52619e99ba338b8c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc) 2024-02-07T13:12:06.748 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:12:06 smithi191 podman[46051]: 2024-02-07 13:12:06.449437966 +0000 UTC m=+0.385504773 container init 156c69648757d77f05fb9292fcf1548e10ce632f81d18c52619e99ba338b8c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc) 2024-02-07T13:12:06.748 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:12:06 smithi191 podman[46051]: 2024-02-07 13:12:06.455758879 +0000 UTC m=+0.391825682 container start 156c69648757d77f05fb9292fcf1548e10ce632f81d18c52619e99ba338b8c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-2-deactivate, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-02-07T13:12:06.748 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:12:06 smithi191 podman[46051]: 2024-02-07 13:12:06.532541627 +0000 UTC m=+0.468608430 container attach 156c69648757d77f05fb9292fcf1548e10ce632f81d18c52619e99ba338b8c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-02-07T13:12:07.018 INFO:journalctl@ceph.osd.2.smithi191.stdout:Feb 07 13:12:06 smithi191 podman[46051]: 2024-02-07 13:12:06.748796006 +0000 UTC m=+0.684862816 container died 156c69648757d77f05fb9292fcf1548e10ce632f81d18c52619e99ba338b8c64 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-07T13:12:07.304 DEBUG:teuthology.orchestra.run.smithi191:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@osd.2.service' 2024-02-07T13:12:07.345 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:12:07.345 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-07T13:12:07.345 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-07T13:12:07.345 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@rgw.foo.a 2024-02-07T13:12:07.657 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:Feb 07 13:12:07 smithi067 systemd[1]: Stopping Ceph rgw.foo.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:12:08.055 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:Feb 07 13:12:07 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-rgw-foo-a[41052]: 2024-02-07T13:12:07.657+0000 7fd411ada700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-07T13:12:08.055 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:Feb 07 13:12:07 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-rgw-foo-a[41052]: 2024-02-07T13:12:07.657+0000 7fd420345ac0 -1 shutting down 2024-02-07T13:12:17.964 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:Feb 07 13:12:17 smithi067 bash[54512]: time="2024-02-07T13:12:17Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-02-07T13:12:17.965 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:Feb 07 13:12:17 smithi067 podman[54512]: 2024-02-07 13:12:17.6846712 +0000 UTC m=+10.140861996 container died d68d5b811d4b69d833b9e91af9718a5bb8870b4de85d699041a69aeb163ce765 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-rgw-foo-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-02-07T13:12:18.305 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:Feb 07 13:12:18 smithi067 podman[54512]: 2024-02-07 13:12:18.146029194 +0000 UTC m=+10.602219992 container cleanup d68d5b811d4b69d833b9e91af9718a5bb8870b4de85d699041a69aeb163ce765 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:c53d1d048facb107ddbb5178fea4d5ae496b0f9f9f34c5f63d4bece0be7a8bbe, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-rgw-foo-a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240131, RELEASE=wip-guits-testing-4-2024-02-05-1854-44458cc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, ceph=True) 2024-02-07T13:12:18.306 INFO:journalctl@ceph.rgw.foo.a.smithi067.stdout:Feb 07 13:12:18 smithi067 bash[54512]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-rgw-foo-a 2024-02-07T13:12:18.568 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@rgw.foo.a.service' 2024-02-07T13:12:18.605 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:12:18.605 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-07T13:12:18.606 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-07T13:12:18.606 DEBUG:teuthology.orchestra.run.smithi159:> sudo systemctl stop ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@prometheus.a 2024-02-07T13:12:18.905 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:12:18 smithi159 systemd[1]: Stopping Ceph prometheus.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:12:18.905 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:12:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:12:18.717Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.67:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.67:8765: connect: connection refused" 2024-02-07T13:12:18.905 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:12:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:12:18.718Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.67:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.67:8765: connect: connection refused" 2024-02-07T13:12:18.905 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:12:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:12:18.904Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-07T13:12:18.905 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:12:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:12:18.905Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-07T13:12:18.905 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:12:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:12:18.905Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-07T13:12:19.193 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:12:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:12:18.905Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-07T13:12:19.193 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:12:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:12:18.905Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-07T13:12:19.193 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:12:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:12:18.905Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-07T13:12:19.193 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:12:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:12:18.905Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-07T13:12:19.193 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:12:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:12:18.905Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-07T13:12:19.194 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:12:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:12:18.905Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-07T13:12:19.194 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:12:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:12:18.905Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-07T13:12:19.194 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:12:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:12:18.905Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-07T13:12:19.194 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:12:18 smithi159 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a[36040]: ts=2024-02-07T13:12:18.905Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-07T13:12:19.194 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:12:18 smithi159 podman[47853]: 2024-02-07 13:12:18.915699221 +0000 UTC m=+0.110263377 container died 0ee24705a6619775c313bc831ea573497aa3daacd376710856f4fdba46299acf (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-07T13:12:19.693 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:12:19 smithi159 podman[47853]: 2024-02-07 13:12:19.29411685 +0000 UTC m=+0.488681005 container cleanup 0ee24705a6619775c313bc831ea573497aa3daacd376710856f4fdba46299acf (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-07T13:12:19.693 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:12:19 smithi159 bash[47853]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a 2024-02-07T13:12:19.693 INFO:journalctl@ceph.prometheus.a.smithi159.stdout:Feb 07 13:12:19 smithi159 podman[47871]: 2024-02-07 13:12:19.453510427 +0000 UTC m=+0.541679116 container remove 0ee24705a6619775c313bc831ea573497aa3daacd376710856f4fdba46299acf (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-07T13:12:19.854 DEBUG:teuthology.orchestra.run.smithi159:> sudo pkill -f 'journalctl -f -n 0 -u ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@prometheus.a.service' 2024-02-07T13:12:19.892 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-07T13:12:19.892 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-07T13:12:19.893 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 --force --keep-logs 2024-02-07T13:12:20.099 INFO:teuthology.orchestra.run.smithi067.stdout:Deleting cluster with fsid: 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:12:21.305 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:12:21 smithi067 systemd[1]: Stopping Ceph alertmanager.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:12:21.693 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:12:21 smithi067 ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-alertmanager-a[47736]: ts=2024-02-07T13:12:21.411Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-07T13:12:21.694 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:12:21 smithi067 podman[55048]: 2024-02-07 13:12:21.432538062 +0000 UTC m=+0.127619551 container died 017b9537c5bc3a4fd6a7c852f9551812312bbf65141c13b85d21f13570d4c359 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-07T13:12:22.056 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:12:21 smithi067 podman[55048]: 2024-02-07 13:12:21.892856195 +0000 UTC m=+0.587937682 container cleanup 017b9537c5bc3a4fd6a7c852f9551812312bbf65141c13b85d21f13570d4c359 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-07T13:12:22.056 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:12:21 smithi067 bash[55048]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-alertmanager-a 2024-02-07T13:12:22.556 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:12:22 smithi067 podman[55067]: 2024-02-07 13:12:22.13565261 +0000 UTC m=+0.702661801 container remove 017b9537c5bc3a4fd6a7c852f9551812312bbf65141c13b85d21f13570d4c359 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-07T13:12:22.556 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:12:22 smithi067 podman[55067]: 2024-02-07 13:12:22.211158162 +0000 UTC m=+0.778167353 volume remove 15101baa307ef5aa9d6c77b476594c24a3308adb3f3754062f23649a2c5b9ed4 2024-02-07T13:12:22.556 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:12:22 smithi067 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-07T13:12:22.556 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:12:22 smithi067 systemd[1]: Stopped Ceph alertmanager.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:12:22.556 INFO:journalctl@ceph.alertmanager.a.smithi067.stdout:Feb 07 13:12:22 smithi067 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@alertmanager.a.service: Consumed 1.062s CPU time. 2024-02-07T13:12:22.819 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:12:22 smithi067 systemd[1]: Stopping Ceph node-exporter.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:12:23.232 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:12:22 smithi067 podman[55184]: 2024-02-07 13:12:22.819045683 +0000 UTC m=+0.114891300 container died 81c9b873491c45336fa5fc877c9b4ec223af1fcf1c15629bc3bca9dda59ea34c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-07T13:12:23.555 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:12:23 smithi067 podman[55184]: 2024-02-07 13:12:23.405943692 +0000 UTC m=+0.701789305 container cleanup 81c9b873491c45336fa5fc877c9b4ec223af1fcf1c15629bc3bca9dda59ea34c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-07T13:12:23.555 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:12:23 smithi067 bash[55184]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a 2024-02-07T13:12:23.968 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:12:23 smithi067 podman[55203]: 2024-02-07 13:12:23.665572926 +0000 UTC m=+0.851904469 container remove 81c9b873491c45336fa5fc877c9b4ec223af1fcf1c15629bc3bca9dda59ea34c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-07T13:12:23.968 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:12:23 smithi067 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-07T13:12:23.968 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:12:23 smithi067 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-07T13:12:23.969 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:12:23 smithi067 systemd[1]: Stopped Ceph node-exporter.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:12:23.969 INFO:journalctl@ceph.node-exporter.a.smithi067.stdout:Feb 07 13:12:23 smithi067 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@node-exporter.a.service: Consumed 1.837s CPU time. 2024-02-07T13:12:24.437 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 --force --keep-logs 2024-02-07T13:12:24.640 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:12:25.908 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:12:25 smithi159 systemd[1]: Stopping Ceph grafana.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:12:26.168 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:12:25 smithi159 podman[48511]: 2024-02-07 13:12:25.906556369 +0000 UTC m=+0.098700008 container died 1eb011bebf11914ce2d9ef7b5274d9872d0d0eb054609ea53bfab0941ae4ea76 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, description=Ceph Grafana Container, distribution-scope=public, vcs-type=git, maintainer=Paul Cuzner , com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.tags=base rhel8, architecture=x86_64, version=8.5, io.openshift.expose-services=, vendor=Red Hat, Inc., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.component=ubi8-container, name=ubi8, release=239.1651231664, io.buildah.version=1.32.0, build-date=2022-04-29T11:48:11.055897, io.k8s.display-name=Red Hat Universal Base Image 8) 2024-02-07T13:12:26.443 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:12:26 smithi159 podman[48511]: 2024-02-07 13:12:26.234206435 +0000 UTC m=+0.426350076 container cleanup 1eb011bebf11914ce2d9ef7b5274d9872d0d0eb054609ea53bfab0941ae4ea76 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a, vcs-type=git, com.redhat.component=ubi8-container, io.buildah.version=1.32.0, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.expose-services=, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, name=ubi8, maintainer=Paul Cuzner , release=239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, architecture=x86_64, distribution-scope=public, vendor=Red Hat, Inc., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, version=8.5, summary=Grafana Container configured for Ceph mgr/dashboard integration, description=Ceph Grafana Container) 2024-02-07T13:12:26.443 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:12:26 smithi159 bash[48511]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a 2024-02-07T13:12:26.443 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:12:26 smithi159 podman[48530]: 2024-02-07 13:12:26.410353396 +0000 UTC m=+0.503447664 container remove 1eb011bebf11914ce2d9ef7b5274d9872d0d0eb054609ea53bfab0941ae4ea76 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-grafana-a, name=ubi8, io.k8s.display-name=Red Hat Universal Base Image 8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, architecture=x86_64, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, maintainer=Paul Cuzner , io.buildah.version=1.32.0, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.expose-services=, build-date=2022-04-29T11:48:11.055897, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-type=git, description=Ceph Grafana Container, com.redhat.component=ubi8-container, distribution-scope=public, version=8.5, release=239.1651231664, vendor=Red Hat, Inc., summary=Grafana Container configured for Ceph mgr/dashboard integration) 2024-02-07T13:12:26.443 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:12:26 smithi159 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-02-07T13:12:26.943 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:12:26 smithi159 systemd[1]: Stopping Ceph node-exporter.b for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:12:26.943 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:12:26 smithi159 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2024-02-07T13:12:26.943 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:12:26 smithi159 systemd[1]: Stopped Ceph grafana.a for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:12:26.943 INFO:journalctl@ceph.grafana.a.smithi159.stdout:Feb 07 13:12:26 smithi159 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@grafana.a.service: Consumed 4.134s CPU time. 2024-02-07T13:12:27.414 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:12:27 smithi159 podman[48646]: 2024-02-07 13:12:27.159959732 +0000 UTC m=+0.169503172 container died 5030b8182be2fa8900cd73317de7c992387473fd74f6381a22924e92211332cd (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-07T13:12:27.693 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:12:27 smithi159 podman[48646]: 2024-02-07 13:12:27.504527989 +0000 UTC m=+0.514071428 container cleanup 5030b8182be2fa8900cd73317de7c992387473fd74f6381a22924e92211332cd (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-07T13:12:27.693 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:12:27 smithi159 bash[48646]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b 2024-02-07T13:12:28.071 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:12:27 smithi159 podman[48665]: 2024-02-07 13:12:27.780575177 +0000 UTC m=+0.625435896 container remove 5030b8182be2fa8900cd73317de7c992387473fd74f6381a22924e92211332cd (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-07T13:12:28.071 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:12:27 smithi159 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-07T13:12:28.071 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:12:27 smithi159 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-07T13:12:28.071 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:12:27 smithi159 systemd[1]: Stopped Ceph node-exporter.b for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:12:28.071 INFO:journalctl@ceph.node-exporter.b.smithi159.stdout:Feb 07 13:12:27 smithi159 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@node-exporter.b.service: Consumed 1.813s CPU time. 2024-02-07T13:12:28.571 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 --force --keep-logs 2024-02-07T13:12:28.765 INFO:teuthology.orchestra.run.smithi191.stdout:Deleting cluster with fsid: 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:12:29.849 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:12:29 smithi191 systemd[1]: Stopping Ceph node-exporter.c for 81df3e5a-c5b9-11ee-95b6-87774f69a715... 2024-02-07T13:12:30.118 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:12:29 smithi191 podman[46892]: 2024-02-07 13:12:29.848333115 +0000 UTC m=+0.119213451 container died 6ca43b66939e74a145ee2ffdfa8eab204acdd532f60989bba944e276a1969799 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-07T13:12:30.486 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:12:30 smithi191 podman[46892]: 2024-02-07 13:12:30.259716711 +0000 UTC m=+0.530597050 container cleanup 6ca43b66939e74a145ee2ffdfa8eab204acdd532f60989bba944e276a1969799 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-07T13:12:30.486 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:12:30 smithi191 bash[46892]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c 2024-02-07T13:12:30.750 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:12:30 smithi191 podman[46911]: 2024-02-07 13:12:30.510965306 +0000 UTC m=+0.662124542 container remove 6ca43b66939e74a145ee2ffdfa8eab204acdd532f60989bba944e276a1969799 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-07T13:12:30.751 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:12:30 smithi191 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@node-exporter.c.service: Main process exited, code=exited, status=143/n/a 2024-02-07T13:12:30.751 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:12:30 smithi191 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@node-exporter.c.service: Failed with result 'exit-code'. 2024-02-07T13:12:30.751 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:12:30 smithi191 systemd[1]: Stopped Ceph node-exporter.c for 81df3e5a-c5b9-11ee-95b6-87774f69a715. 2024-02-07T13:12:30.751 INFO:journalctl@ceph.node-exporter.c.smithi191.stdout:Feb 07 13:12:30 smithi191 systemd[1]: ceph-81df3e5a-c5b9-11ee-95b6-87774f69a715@node-exporter.c.service: Consumed 1.895s CPU time. 2024-02-07T13:12:31.371 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-07T13:12:31.406 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-07T13:12:31.441 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-07T13:12:31.471 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-07T13:12:31.473 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548780/remote/smithi067/crash 2024-02-07T13:12:31.473 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/crash -- . 2024-02-07T13:12:31.513 INFO:teuthology.orchestra.run.smithi067.stderr:tar: /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/crash: Cannot open: No such file or directory 2024-02-07T13:12:31.514 INFO:teuthology.orchestra.run.smithi067.stderr:tar: Error is not recoverable: exiting now 2024-02-07T13:12:31.515 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548780/remote/smithi159/crash 2024-02-07T13:12:31.516 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/crash -- . 2024-02-07T13:12:31.563 INFO:teuthology.orchestra.run.smithi159.stderr:tar: /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/crash: Cannot open: No such file or directory 2024-02-07T13:12:31.563 INFO:teuthology.orchestra.run.smithi159.stderr:tar: Error is not recoverable: exiting now 2024-02-07T13:12:31.565 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/crash to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548780/remote/smithi191/crash 2024-02-07T13:12:31.566 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/crash -- . 2024-02-07T13:12:31.601 INFO:teuthology.orchestra.run.smithi191.stderr:tar: /var/lib/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/crash: Cannot open: No such file or directory 2024-02-07T13:12:31.601 INFO:teuthology.orchestra.run.smithi191.stderr:tar: Error is not recoverable: exiting now 2024-02-07T13:12:31.602 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-07T13:12:31.602 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-07T13:12:31.634 INFO:teuthology.orchestra.run.smithi067.stdout:2024-02-07T13:10:00.000130+0000 mon.a (mon.0) 610 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-07T13:12:31.634 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-07T13:12:31.635 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[SEC\]' /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-07T13:12:31.710 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[ERR\]' /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-07T13:12:31.781 DEBUG:teuthology.orchestra.run.smithi067:> sudo egrep '\[WRN\]' /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-07T13:12:31.848 INFO:teuthology.orchestra.run.smithi067.stdout:2024-02-07T13:10:00.000130+0000 mon.a (mon.0) 610 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-07T13:12:31.848 INFO:tasks.cephadm:Compressing logs... 2024-02-07T13:12:31.849 DEBUG:teuthology.orchestra.run.smithi067:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-07T13:12:31.891 DEBUG:teuthology.orchestra.run.smithi159:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-07T13:12:31.894 DEBUG:teuthology.orchestra.run.smithi191:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-07T13:12:31.915 INFO:teuthology.orchestra.run.smithi067.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-07T13:12:31.915 INFO:teuthology.orchestra.run.smithi067.stderr:: No such file or directory 2024-02-07T13:12:31.915 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-mon.a.log 2024-02-07T13:12:31.915 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.audit.log 2024-02-07T13:12:31.917 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-07T13:12:31.917 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-mgr.y.log 2024-02-07T13:12:31.917 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-osd.0.log 2024-02-07T13:12:31.917 INFO:teuthology.orchestra.run.smithi067.stderr: 57.6% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-07T13:12:31.920 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.cephadm.log 2024-02-07T13:12:31.920 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.log 2024-02-07T13:12:31.920 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-volume.log 2024-02-07T13:12:31.920 INFO:teuthology.orchestra.run.smithi067.stderr: 82.8% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.cephadm.log.gz 89.9% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.audit.log.gz 2024-02-07T13:12:31.920 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-07T13:12:31.920 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.log: /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-volume.log: 85.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-07T13:12:31.920 INFO:teuthology.orchestra.run.smithi067.stderr: 85.0% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.log.gz 2024-02-07T13:12:31.921 INFO:teuthology.orchestra.run.smithi191.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-07T13:12:31.921 INFO:teuthology.orchestra.run.smithi191.stderr:: No such file or directory 2024-02-07T13:12:31.921 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.audit.log 2024-02-07T13:12:31.921 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-mgr.z.log 2024-02-07T13:12:31.922 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.cephadm.log 2024-02-07T13:12:31.925 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.log 2024-02-07T13:12:31.925 INFO:teuthology.orchestra.run.smithi191.stderr:gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-osd.2.log 2024-02-07T13:12:31.925 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.cephadm.log: /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-volume.log 2024-02-07T13:12:31.925 INFO:teuthology.orchestra.run.smithi191.stderr: 81.5% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.cephadm.log.gz 2024-02-07T13:12:31.925 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-mon.c.log 2024-02-07T13:12:31.926 INFO:teuthology.orchestra.run.smithi191.stderr: 90.0% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.audit.log.gz 2024-02-07T13:12:31.926 INFO:teuthology.orchestra.run.smithi191.stderr: 80.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-07T13:12:31.926 INFO:teuthology.orchestra.run.smithi191.stderr: 84.7% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.log.gz 2024-02-07T13:12:31.926 INFO:teuthology.orchestra.run.smithi159.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-07T13:12:31.927 INFO:teuthology.orchestra.run.smithi159.stderr:: No such file or directory 2024-02-07T13:12:31.927 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.audit.log 2024-02-07T13:12:31.927 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-osd.1.log 2024-02-07T13:12:31.927 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-mgr.x.log 2024-02-07T13:12:31.927 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.cephadm.log 2024-02-07T13:12:31.927 INFO:teuthology.orchestra.run.smithi159.stderr:gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.log 2024-02-07T13:12:31.927 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-mgr.x.log: /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-volume.log 2024-02-07T13:12:31.927 INFO:teuthology.orchestra.run.smithi159.stderr: 89.9% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.audit.log.gz 2024-02-07T13:12:31.927 INFO:teuthology.orchestra.run.smithi159.stderr: 81.5% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.cephadm.log.gz 2024-02-07T13:12:31.927 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-mon.b.log 2024-02-07T13:12:31.927 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-volume.log: 84.6% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph.log.gz 2024-02-07T13:12:31.927 INFO:teuthology.orchestra.run.smithi159.stderr:/var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-mon.b.log: 77.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-07T13:12:31.928 INFO:teuthology.orchestra.run.smithi191.stderr:/var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-volume.log: /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-mon.c.log: 90.3% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-mgr.z.log.gz 2024-02-07T13:12:31.929 INFO:teuthology.orchestra.run.smithi159.stderr: 90.3% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-mgr.x.log.gz 2024-02-07T13:12:31.939 INFO:teuthology.orchestra.run.smithi191.stderr: 90.2% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-volume.log.gz 2024-02-07T13:12:31.943 INFO:teuthology.orchestra.run.smithi067.stderr: 90.1% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-volume.log.gz 2024-02-07T13:12:31.951 INFO:teuthology.orchestra.run.smithi159.stderr: 90.1% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-volume.log.gz 2024-02-07T13:12:31.984 INFO:teuthology.orchestra.run.smithi191.stderr: 92.4% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-mon.c.log.gz 2024-02-07T13:12:31.987 INFO:teuthology.orchestra.run.smithi159.stderr: 92.3% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-mon.b.log.gz 2024-02-07T13:12:32.034 INFO:teuthology.orchestra.run.smithi067.stderr: 88.1% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-mgr.y.log.gz 2024-02-07T13:12:32.136 INFO:teuthology.orchestra.run.smithi067.stderr: 91.5% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-mon.a.log.gz 2024-02-07T13:12:32.197 INFO:teuthology.orchestra.run.smithi191.stderr: 94.3% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-osd.2.log.gz 2024-02-07T13:12:32.199 INFO:teuthology.orchestra.run.smithi191.stderr: 2024-02-07T13:12:32.199 INFO:teuthology.orchestra.run.smithi191.stderr:real 0m0.291s 2024-02-07T13:12:32.199 INFO:teuthology.orchestra.run.smithi191.stderr:user 0m0.358s 2024-02-07T13:12:32.199 INFO:teuthology.orchestra.run.smithi191.stderr:sys 0m0.036s 2024-02-07T13:12:32.204 INFO:teuthology.orchestra.run.smithi067.stderr: 94.3% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-osd.0.log.gz 2024-02-07T13:12:32.206 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-02-07T13:12:32.206 INFO:teuthology.orchestra.run.smithi067.stderr:real 0m0.304s 2024-02-07T13:12:32.207 INFO:teuthology.orchestra.run.smithi067.stderr:user 0m0.638s 2024-02-07T13:12:32.207 INFO:teuthology.orchestra.run.smithi067.stderr:sys 0m0.051s 2024-02-07T13:12:32.249 INFO:teuthology.orchestra.run.smithi159.stderr: 94.3% -- replaced with /var/log/ceph/81df3e5a-c5b9-11ee-95b6-87774f69a715/ceph-osd.1.log.gz 2024-02-07T13:12:32.251 INFO:teuthology.orchestra.run.smithi159.stderr: 2024-02-07T13:12:32.251 INFO:teuthology.orchestra.run.smithi159.stderr:real 0m0.343s 2024-02-07T13:12:32.251 INFO:teuthology.orchestra.run.smithi159.stderr:user 0m0.427s 2024-02-07T13:12:32.251 INFO:teuthology.orchestra.run.smithi159.stderr:sys 0m0.034s 2024-02-07T13:12:32.252 INFO:tasks.cephadm:Archiving logs... 2024-02-07T13:12:32.252 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548780/remote/smithi067/log 2024-02-07T13:12:32.253 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-07T13:12:32.382 DEBUG:teuthology.misc:Transferring archived files from smithi159:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548780/remote/smithi159/log 2024-02-07T13:12:32.383 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-07T13:12:32.480 DEBUG:teuthology.misc:Transferring archived files from smithi191:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548780/remote/smithi191/log 2024-02-07T13:12:32.481 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-07T13:12:32.564 INFO:tasks.cephadm:Removing cluster... 2024-02-07T13:12:32.564 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 --force 2024-02-07T13:12:32.747 INFO:teuthology.orchestra.run.smithi067.stdout:Deleting cluster with fsid: 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:12:32.864 DEBUG:teuthology.orchestra.run.smithi159:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 --force 2024-02-07T13:12:33.044 INFO:teuthology.orchestra.run.smithi159.stdout:Deleting cluster with fsid: 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:12:33.158 DEBUG:teuthology.orchestra.run.smithi191:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 81df3e5a-c5b9-11ee-95b6-87774f69a715 --force 2024-02-07T13:12:33.339 INFO:teuthology.orchestra.run.smithi191.stdout:Deleting cluster with fsid: 81df3e5a-c5b9-11ee-95b6-87774f69a715 2024-02-07T13:12:33.462 INFO:tasks.cephadm:Removing cephadm ... 2024-02-07T13:12:33.462 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-07T13:12:33.478 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-07T13:12:33.494 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-07T13:12:33.508 INFO:tasks.cephadm:Teardown complete 2024-02-07T13:12:33.509 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2024-02-07T13:12:33.518 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi067:/dev/vg_nvme/lv_1... 2024-02-07T13:12:33.518 DEBUG:teuthology.orchestra.run.smithi067:> sudo nvme disconnect -n lv_1 2024-02-07T13:12:33.671 INFO:teuthology.orchestra.run.smithi067.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-07T13:12:33.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:12:33.673 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi067:/dev/vg_nvme/lv_2... 2024-02-07T13:12:33.673 DEBUG:teuthology.orchestra.run.smithi067:> sudo nvme disconnect -n lv_2 2024-02-07T13:12:33.811 INFO:teuthology.orchestra.run.smithi067.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-07T13:12:33.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:12:33.813 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi067:/dev/vg_nvme/lv_3... 2024-02-07T13:12:33.813 DEBUG:teuthology.orchestra.run.smithi067:> sudo nvme disconnect -n lv_3 2024-02-07T13:12:33.960 INFO:teuthology.orchestra.run.smithi067.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-07T13:12:33.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:12:33.962 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi067:/dev/vg_nvme/lv_4... 2024-02-07T13:12:33.962 DEBUG:teuthology.orchestra.run.smithi067:> sudo nvme disconnect -n lv_4 2024-02-07T13:12:34.115 INFO:teuthology.orchestra.run.smithi067.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-07T13:12:34.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:12:34.116 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-02-07T13:12:34.116 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/scratch_devs 2024-02-07T13:12:34.143 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi159:/dev/vg_nvme/lv_1... 2024-02-07T13:12:34.143 DEBUG:teuthology.orchestra.run.smithi159:> sudo nvme disconnect -n lv_1 2024-02-07T13:12:34.297 INFO:teuthology.orchestra.run.smithi159.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-07T13:12:34.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:12:34.300 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi159:/dev/vg_nvme/lv_2... 2024-02-07T13:12:34.300 DEBUG:teuthology.orchestra.run.smithi159:> sudo nvme disconnect -n lv_2 2024-02-07T13:12:34.456 INFO:teuthology.orchestra.run.smithi159.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-07T13:12:34.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:12:34.458 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi159:/dev/vg_nvme/lv_3... 2024-02-07T13:12:34.458 DEBUG:teuthology.orchestra.run.smithi159:> sudo nvme disconnect -n lv_3 2024-02-07T13:12:34.607 INFO:teuthology.orchestra.run.smithi159.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-07T13:12:34.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:12:34.609 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi159:/dev/vg_nvme/lv_4... 2024-02-07T13:12:34.609 DEBUG:teuthology.orchestra.run.smithi159:> sudo nvme disconnect -n lv_4 2024-02-07T13:12:34.763 INFO:teuthology.orchestra.run.smithi159.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-07T13:12:34.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:12:34.765 DEBUG:teuthology.orchestra.run.smithi159:> set -ex 2024-02-07T13:12:34.765 DEBUG:teuthology.orchestra.run.smithi159:> sudo dd of=/scratch_devs 2024-02-07T13:12:34.795 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi191:/dev/vg_nvme/lv_1... 2024-02-07T13:12:34.795 DEBUG:teuthology.orchestra.run.smithi191:> sudo nvme disconnect -n lv_1 2024-02-07T13:12:34.931 INFO:teuthology.orchestra.run.smithi191.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-07T13:12:34.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:12:34.932 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi191:/dev/vg_nvme/lv_2... 2024-02-07T13:12:34.933 DEBUG:teuthology.orchestra.run.smithi191:> sudo nvme disconnect -n lv_2 2024-02-07T13:12:35.062 INFO:teuthology.orchestra.run.smithi191.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-07T13:12:35.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:12:35.064 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi191:/dev/vg_nvme/lv_3... 2024-02-07T13:12:35.064 DEBUG:teuthology.orchestra.run.smithi191:> sudo nvme disconnect -n lv_3 2024-02-07T13:12:35.287 INFO:teuthology.orchestra.run.smithi191.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-07T13:12:35.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:12:35.289 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi191:/dev/vg_nvme/lv_4... 2024-02-07T13:12:35.289 DEBUG:teuthology.orchestra.run.smithi191:> sudo nvme disconnect -n lv_4 2024-02-07T13:12:35.449 INFO:teuthology.orchestra.run.smithi191.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-07T13:12:35.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:12:35.451 DEBUG:teuthology.orchestra.run.smithi191:> set -ex 2024-02-07T13:12:35.451 DEBUG:teuthology.orchestra.run.smithi191:> sudo dd of=/scratch_devs 2024-02-07T13:12:35.478 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-07T13:12:35.488 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-07T13:12:35.488 DEBUG:teuthology.orchestra.run.smithi067:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-07T13:12:35.491 DEBUG:teuthology.orchestra.run.smithi159:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-07T13:12:35.493 DEBUG:teuthology.orchestra.run.smithi191:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-07T13:12:35.505 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2024-02-07T13:12:35.507 INFO:teuthology.orchestra.run.smithi159.stderr:bash: line 1: ntpq: command not found 2024-02-07T13:12:35.510 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-07T13:12:35.510 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-02-07T13:12:35.510 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 1 -780us[ -780us] +/- 89ms 2024-02-07T13:12:35.510 INFO:teuthology.orchestra.run.smithi067.stdout:^* hv02.front.sepia.ceph.com 3 6 377 2 +307us[ +306us] +/- 49ms 2024-02-07T13:12:35.510 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 +242us[ +242us] +/- 85ms 2024-02-07T13:12:35.510 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:12:35.512 INFO:teuthology.orchestra.run.smithi159.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-07T13:12:35.512 INFO:teuthology.orchestra.run.smithi159.stdout:=============================================================================== 2024-02-07T13:12:35.512 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 1 -782us[ -782us] +/- 89ms 2024-02-07T13:12:35.512 INFO:teuthology.orchestra.run.smithi159.stdout:^* hv02.front.sepia.ceph.com 3 6 377 1 +311us[ +308us] +/- 49ms 2024-02-07T13:12:35.512 INFO:teuthology.orchestra.run.smithi159.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +227us[ +224us] +/- 85ms 2024-02-07T13:12:35.512 INFO:teuthology.orchestra.run.smithi159.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:12:35.532 INFO:teuthology.orchestra.run.smithi191.stderr:bash: line 1: ntpq: command not found 2024-02-07T13:12:35.536 INFO:teuthology.orchestra.run.smithi191.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-07T13:12:35.536 INFO:teuthology.orchestra.run.smithi191.stdout:=============================================================================== 2024-02-07T13:12:35.537 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 1 -777us[ -777us] +/- 89ms 2024-02-07T13:12:35.537 INFO:teuthology.orchestra.run.smithi191.stdout:^* hv02.front.sepia.ceph.com 3 6 377 1 +322us[ +327us] +/- 49ms 2024-02-07T13:12:35.537 INFO:teuthology.orchestra.run.smithi191.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 +255us[ +260us] +/- 85ms 2024-02-07T13:12:35.537 INFO:teuthology.orchestra.run.smithi191.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-07T13:12:35.537 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-07T13:12:35.546 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-07T13:12:35.547 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-07T13:12:35.630 DEBUG:teuthology.orchestra.run.smithi067:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-07T13:12:35.710 DEBUG:teuthology.orchestra.run.smithi159:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-07T13:12:35.785 DEBUG:teuthology.orchestra.run.smithi191:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-07T13:12:35.854 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-07T13:12:35.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:12:35.887 DEBUG:teuthology.orchestra.run.smithi159:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-07T13:12:35.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:12:35.920 DEBUG:teuthology.orchestra.run.smithi191:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-07T13:12:35.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:12:35.950 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-07T13:12:35.960 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-07T13:12:35.971 INFO:teuthology.task.internal:Duration was 744.496667 seconds 2024-02-07T13:12:35.972 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-07T13:12:35.981 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-07T13:12:35.981 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-07T13:12:35.984 DEBUG:teuthology.orchestra.run.smithi159:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-07T13:12:35.986 DEBUG:teuthology.orchestra.run.smithi191:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-07T13:12:36.028 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-07T13:12:36.028 INFO:teuthology.orchestra.run.smithi159.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-07T13:12:36.033 INFO:teuthology.orchestra.run.smithi191.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-07T13:12:36.487 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-07T13:12:36.488 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi067.front.sepia.ceph.com 2024-02-07T13:12:36.488 DEBUG:teuthology.orchestra.run.smithi067:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-07T13:12:36.511 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi159.front.sepia.ceph.com 2024-02-07T13:12:36.512 DEBUG:teuthology.orchestra.run.smithi159:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-07T13:12:36.556 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi191.front.sepia.ceph.com 2024-02-07T13:12:36.557 DEBUG:teuthology.orchestra.run.smithi191:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-07T13:12:36.580 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-07T13:12:36.580 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-07T13:12:36.582 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-07T13:12:36.599 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-07T13:12:36.648 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-07T13:12:36.649 DEBUG:teuthology.orchestra.run.smithi067:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-07T13:12:36.653 DEBUG:teuthology.orchestra.run.smithi159:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-07T13:12:36.669 DEBUG:teuthology.orchestra.run.smithi191:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-07T13:12:36.848 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-07T13:12:36.860 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-07T13:12:36.860 DEBUG:teuthology.orchestra.run.smithi067:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-07T13:12:36.886 DEBUG:teuthology.orchestra.run.smithi159:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-07T13:12:36.913 DEBUG:teuthology.orchestra.run.smithi191:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-07T13:12:36.941 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-07T13:12:36.951 DEBUG:teuthology.orchestra.run.smithi067:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-07T13:12:36.953 DEBUG:teuthology.orchestra.run.smithi159:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-07T13:12:36.956 DEBUG:teuthology.orchestra.run.smithi191:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-07T13:12:36.976 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = core 2024-02-07T13:12:36.978 INFO:teuthology.orchestra.run.smithi159.stdout:kernel.core_pattern = core 2024-02-07T13:12:37.006 INFO:teuthology.orchestra.run.smithi191.stdout:kernel.core_pattern = core 2024-02-07T13:12:37.056 DEBUG:teuthology.orchestra.run.smithi067:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-07T13:12:37.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:12:37.070 DEBUG:teuthology.orchestra.run.smithi159:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-07T13:12:37.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:12:37.084 DEBUG:teuthology.orchestra.run.smithi191:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-07T13:12:37.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-07T13:12:37.110 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-07T13:12:37.122 INFO:teuthology.task.internal:Transferring archived files... 2024-02-07T13:12:37.122 DEBUG:teuthology.misc:Transferring archived files from smithi067:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548780/remote/smithi067 2024-02-07T13:12:37.123 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-07T13:12:37.162 DEBUG:teuthology.misc:Transferring archived files from smithi159:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548780/remote/smithi159 2024-02-07T13:12:37.163 DEBUG:teuthology.orchestra.run.smithi159:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-07T13:12:37.200 DEBUG:teuthology.misc:Transferring archived files from smithi191:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548780/remote/smithi191 2024-02-07T13:12:37.201 DEBUG:teuthology.orchestra.run.smithi191:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-07T13:12:37.238 INFO:teuthology.task.internal:Removing archive directory... 2024-02-07T13:12:37.238 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-07T13:12:37.241 DEBUG:teuthology.orchestra.run.smithi159:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-07T13:12:37.243 DEBUG:teuthology.orchestra.run.smithi191:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-07T13:12:37.288 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-07T13:12:37.301 INFO:teuthology.task.internal:Not uploading archives. 2024-02-07T13:12:37.301 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-07T13:12:37.311 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-07T13:12:37.311 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-07T13:12:37.313 DEBUG:teuthology.orchestra.run.smithi159:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-07T13:12:37.316 DEBUG:teuthology.orchestra.run.smithi191:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-07T13:12:37.325 INFO:teuthology.orchestra.run.smithi067.stdout: 265293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 7 13:12 /home/ubuntu/cephtest 2024-02-07T13:12:37.327 INFO:teuthology.orchestra.run.smithi159.stdout: 265293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 7 13:12 /home/ubuntu/cephtest 2024-02-07T13:12:37.342 INFO:teuthology.orchestra.run.smithi191.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 7 13:12 /home/ubuntu/cephtest 2024-02-07T13:12:37.343 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-07T13:12:37.354 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-07T13:12:37.389 INFO:teuthology.nuke:Checking targets against current locks 2024-02-07T13:12:37.417 DEBUG:teuthology.nuke:shortname: smithi067 2024-02-07T13:12:37.418 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-07T13:12:37.440 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548780', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-07 12:50:36.450458', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBbRfxaR4TLBQUqZhdNEB6Zyt9efg308UYaaj47aM8raZaJe3NYGHgH0VB44jyLnElgoTBPRQUvpC9Qx6Vez9m0='} 2024-02-07T13:12:37.445 DEBUG:teuthology.nuke:shortname: smithi159 2024-02-07T13:12:37.445 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-07T13:12:37.468 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi159.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548780', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-07 12:50:36.448735', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHEw+djB8EJKcw0sOA9wPa/ZFIMr9kOVaW4Bfd9wkLaWufbicaEeT7nXXKB/tC91Vi7JM9NEIshAX0gmtK6WUKo='} 2024-02-07T13:12:37.469 DEBUG:teuthology.nuke:shortname: smithi191 2024-02-07T13:12:37.469 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-07T13:12:37.491 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi191.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-02-06_08:19:47-orch:cephadm-wip-guits-testing-4-2024-02-05-1854-distro-default-smithi/7548780', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-07 12:50:36.451584', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCgTAM8Rvcs7WzdlG747Si/SsWJQLrCEYd9wwNF1ludTA6s1fYN91CIQrbTwl2pneB4lCLe6eP72uSWvzz7sw1g='} 2024-02-07T13:12:37.501 INFO:teuthology.orchestra.console.smithi067:Power off 2024-02-07T13:12:37.502 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-07T13:12:37.526 DEBUG:teuthology.orchestra.console.smithi067:power off output: Chassis Power Control: Down/Off 2024-02-07T13:12:37.526 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:12:37.541 INFO:teuthology.orchestra.console.smithi159:Power off 2024-02-07T13:12:37.541 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-07T13:12:37.555 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2024-02-07T13:12:37.563 DEBUG:teuthology.orchestra.console.smithi159:power off output: Chassis Power Control: Down/Off 2024-02-07T13:12:37.563 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:12:37.578 INFO:teuthology.orchestra.console.smithi191:Power off 2024-02-07T13:12:37.578 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-07T13:12:37.588 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2024-02-07T13:12:37.601 DEBUG:teuthology.orchestra.console.smithi191:power off output: Chassis Power Control: Down/Off 2024-02-07T13:12:37.601 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:12:37.623 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-02-07T13:12:41.556 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:12:41.589 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:12:41.624 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:12:41.666 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2024-02-07T13:12:41.699 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is on 2024-02-07T13:12:41.733 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-02-07T13:12:45.671 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:12:45.699 DEBUG:teuthology.orchestra.console.smithi159:pexpect command: ipmitool -H smithi159.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:12:45.734 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:12:45.781 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is on 2024-02-07T13:12:45.811 DEBUG:teuthology.orchestra.console.smithi159:check power output: Chassis Power is off 2024-02-07T13:12:45.846 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is on 2024-02-07T13:12:45.912 INFO:teuthology.orchestra.console.smithi159:Power off completed 2024-02-07T13:12:49.786 DEBUG:teuthology.orchestra.console.smithi067:pexpect command: ipmitool -H smithi067.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:12:49.847 DEBUG:teuthology.orchestra.console.smithi191:pexpect command: ipmitool -H smithi191.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-07T13:12:49.896 DEBUG:teuthology.orchestra.console.smithi067:check power output: Chassis Power is off 2024-02-07T13:12:49.956 DEBUG:teuthology.orchestra.console.smithi191:check power output: Chassis Power is off 2024-02-07T13:12:49.998 INFO:teuthology.orchestra.console.smithi067:Power off completed 2024-02-07T13:12:50.058 INFO:teuthology.orchestra.console.smithi191:Power off completed 2024-02-07T13:12:50.162 INFO:teuthology.run:Summary data: description: orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/on fixed-2 mon_election/classic start} duration: 744.4966669082642 failure_reason: '"2024-02-07T13:10:00.000130+0000 mon.a (mon.0) 610 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled" in cluster log' owner: scheduled_gabrioux@teuthology success: false 2024-02-07T13:12:50.162 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-07T13:12:50.240 INFO:teuthology.run:FAIL