2024-06-14T22:20:43.397 INFO:root:teuthology version: 0.0.1.dev316+gc1b06c2 2024-06-14T22:20:43.398 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --description upgrade:quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} --verbose --archive /home/teuthworker/archive/yuriw-2024-06-14_18:07:03-upgrade:quincy-x-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7755958 --name yuriw-2024-06-14_18:07:03-upgrade:quincy-x-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi -- /home/teuthworker/archive/yuriw-2024-06-14_18:07:03-upgrade:quincy-x-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7755958/orig.config.yaml 2024-06-14T22:20:43.449 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-14T22:20:43.530 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-06-14_18:07:03-upgrade:quincy-x-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7755958 branch: wip-yuri5-testing-2024-06-08-0902-reef description: upgrade:quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} email: yweinste@redhat.com first-half-sequence: - cephadm.shell: env: - sha1 mon.a: - ceph config set mgr mgr/cephadm/daemon_cache_timeout 60 - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - ceph orch ps - echo wait for minority of mons to upgrade - while ! ceph mon versions | grep $sha1 ; do sleep 2 ; done - ceph orch ps - ceph orch upgrade pause - sleep 60 - ceph orch upgrade resume - echo wait for majority of mons to upgrade - 'while ! ceph mon versions | grep $sha1 | egrep '': [23]'' ; do sleep 2 ; done' - ceph orch ps - ceph orch upgrade pause - sleep 60 - ceph orch upgrade resume - echo wait for all mons to upgrade - 'while ! ceph mon versions | grep $sha1 | grep '': 3'' ; do sleep 2 ; done' - ceph orch ps - ceph orch upgrade pause - sleep 60 - ceph orch upgrade resume - echo wait for half of osds to upgrade - 'while ! ceph osd versions | grep $sha1 | egrep '': [45678]''; do sleep 2 ; done' - ceph orch upgrade pause - ceph orch ps - ceph orch ps - ceph versions first-half-tasks: - workunit: branch: quincy clients: client.0: - rbd/test_librbd.sh env: RBD_FEATURES: '61' - print: '**** done rbd/test_librbd.sh 7-workload' first_in_suite: false job_id: '7755958' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'librbd C and C++ api tests ' - desc: 'run randomized correctness test for rados operations generate write load with rados bench ' - desc: 'run basic cls tests for rbd ' - desc: 'run basic import/export cli tests for rbd ' - desc: 'librbd C and C++ api tests ' - desc: 'randomized correctness test for rados operations on a replicated pool, using only reads, writes, and deletes ' - desc: 'randomized correctness test for rados operations on a replicated pool with snapshot operations ' - desc: 'run randomized correctness test for rados operations generate write load with rados bench ' name: yuriw-2024-06-14_18:07:03-upgrade:quincy-x-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri5-testing-2024-06-08-0902-reef ceph: conf: global: mon election default strategy: 1 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true create_rbd_pool: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-reef-drop-centos8 sha1: 906f73e382c3916fe5507b8227cc56e63a20794f owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b second-half-sequence: sequential: - cephadm.shell: env: - sha1 mon.a: - ceph orch upgrade resume - sleep 60 - echo wait for upgrade to complete - while ceph orch upgrade status | jq '.in_progress' | grep true ; do ceph orch ps ; ceph versions ; sleep 30 ; done - echo upgrade complete - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 second-half-tasks: - full_sequential: - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - print: '**** done end radosbench.yaml' seed: 5916 sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc sleep_before_teardown: 0 stress-tasks: - thrashosds: aggressive_pg_num_changes: false chance_force_recovery: 0 chance_pgnum_grow: 1 chance_pgpnum_fix: 1 chance_thrash_cluster_full: 0 chance_thrash_pg_upmap: 0 chance_thrash_pg_upmap_items: 0 disable_objectstore_tool_tests: true timeout: 1200 - full_sequential: - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - print: '**** done end radosbench.yaml' - workunit: branch: quincy clients: client.0: - cls/test_cls_rbd.sh - print: '**** done cls/test_cls_rbd.sh 5-workload' - workunit: branch: quincy clients: client.0: - rbd/import_export.sh env: RBD_CREATE_ARGS: --new-format - print: '**** done rbd/import_export.sh 5-workload' - workunit: branch: quincy clients: client.0: - rbd/test_librbd.sh env: RBD_FEATURES: '61' - print: '**** done rbd/test_librbd.sh 7-workload' - full_sequential: - rados: clients: - client.0 objects: 500 op_weights: delete: 10 read: 45 write: 45 ops: 4000 write_append_excl: false - print: '**** done rados/readwrite 5-workload' - full_sequential: - rados: clients: - client.0 objects: 50 op_weights: delete: 50 read: 100 rollback: 50 snap_create: 50 snap_remove: 50 write: 100 ops: 4000 write_append_excl: false - print: '**** done rados/snaps-few-objects 5-workload' subset: 111/120000 suite: upgrade:quincy-x suite_branch: wip-reef-drop-centos8 suite_path: /home/teuthworker/src/github.com_cbodley_ceph_906f73e382c3916fe5507b8227cc56e63a20794f/qa suite_relpath: qa suite_repo: https://github.com/cbodley/ceph.git suite_sha1: 906f73e382c3916fe5507b8227cc56e63a20794f targets: smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIdwvf7xWyWTum7CSE0WRjQV9MBwEycX9EIeorlMLDQL3IODZbx6PDRm96awzY4oWIVutRRB+QlU0aG3ryIl2Zs= smithi178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNIBBDZ+KvYqDxZXZXMmyT7302sX8Abb2ggS1brN0HbvBujfnZ7rDtHQe8Y8M2FTzLXWQRdJ5g+V0csMnL8Bnm4= tasks: - pexec: all: - sudo dnf install runc -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: branch: quincy exclude_packages: - ceph-volume - cephadm: cephadm_branch: quincy cephadm_git_url: https://github.com/ceph/ceph conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.ceph.io/ceph-ci/ceph:quincy - cephadm.shell: mon.a: - ceph fs volume create foo - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph.healthy: null - print: '**** upgrading first half of cluster, with stress ****' - parallel: - first-half-tasks - first-half-sequence - print: '**** done upgrading first half of cluster ****' - ceph.healthy: null - print: '**** applying stress + thrashing to mixed-version cluster ****' - parallel: - stress-tasks - ceph.healthy: null - print: '**** finishing upgrade ****' - parallel: - second-half-tasks - second-half-sequence - ceph.healthy: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: c1b06c2b5d3a02a3583265a115f09e052855bcf2 timestamp: 2024-06-14_18:07:03 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-14T22:20:43.530 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_cbodley_ceph_906f73e382c3916fe5507b8227cc56e63a20794f/qa; will attempt to use it 2024-06-14T22:20:43.531 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_cbodley_ceph_906f73e382c3916fe5507b8227cc56e63a20794f/qa/tasks 2024-06-14T22:20:43.531 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-14T22:20:43.532 INFO:teuthology.task.internal:Checking packages... 2024-06-14T22:20:43.554 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '6f4b09560ed399c3f139402a02f389c2fc8bb2cc' 2024-06-14T22:20:43.554 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-14T22:20:43.554 INFO:teuthology.packaging:ref: None 2024-06-14T22:20:43.554 INFO:teuthology.packaging:tag: None 2024-06-14T22:20:43.554 INFO:teuthology.packaging:branch: wip-yuri5-testing-2024-06-08-0902-reef 2024-06-14T22:20:43.554 INFO:teuthology.packaging:sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-14T22:20:43.554 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri5-testing-2024-06-08-0902-reef 2024-06-14T22:20:43.779 INFO:teuthology.task.internal:Found packages for ceph version 18.2.2-1443.g6f4b0956 2024-06-14T22:20:43.780 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-14T22:20:43.787 INFO:teuthology.task.internal:no buildpackages task found 2024-06-14T22:20:43.787 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-14T22:20:43.847 INFO:teuthology.task.internal:Saving configuration 2024-06-14T22:20:43.867 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-14T22:20:43.875 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-14T22:20:43.897 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-06-14_18:07:03-upgrade:quincy-x-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7755958', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-14 22:15:07.383692', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIdwvf7xWyWTum7CSE0WRjQV9MBwEycX9EIeorlMLDQL3IODZbx6PDRm96awzY4oWIVutRRB+QlU0aG3ryIl2Zs='} 2024-06-14T22:20:43.914 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-06-14_18:07:03-upgrade:quincy-x-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7755958', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-14 22:15:07.382141', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNIBBDZ+KvYqDxZXZXMmyT7302sX8Abb2ggS1brN0HbvBujfnZ7rDtHQe8Y8M2FTzLXWQRdJ5g+V0csMnL8Bnm4='} 2024-06-14T22:20:43.914 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-14T22:20:43.920 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a', 'alertmanager.a'] 2024-06-14T22:20:43.920 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2024-06-14T22:20:43.920 INFO:teuthology.run_tasks:Running task console_log... 2024-06-14T22:20:44.000 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9c49eb7ac0>, signals=[15]) 2024-06-14T22:20:44.000 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-14T22:20:44.006 INFO:teuthology.task.internal:Opening connections... 2024-06-14T22:20:44.007 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2024-06-14T22:20:44.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-14T22:20:44.083 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2024-06-14T22:20:44.084 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-14T22:20:44.153 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-14T22:20:44.159 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2024-06-14T22:20:44.177 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2024-06-14T22:20:44.177 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-06-14T22:20:44.234 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="CentOS Stream" 2024-06-14T22:20:44.234 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="9" 2024-06-14T22:20:44.234 INFO:teuthology.orchestra.run.smithi046.stdout:ID="centos" 2024-06-14T22:20:44.234 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="rhel fedora" 2024-06-14T22:20:44.234 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="9" 2024-06-14T22:20:44.234 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el9" 2024-06-14T22:20:44.234 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-14T22:20:44.234 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2024-06-14T22:20:44.234 INFO:teuthology.orchestra.run.smithi046.stdout:LOGO="fedora-logo-icon" 2024-06-14T22:20:44.235 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-14T22:20:44.235 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://centos.org/" 2024-06-14T22:20:44.235 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-14T22:20:44.235 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-14T22:20:44.235 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-14T22:20:44.235 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-06-14T22:20:44.257 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2024-06-14T22:20:44.275 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2024-06-14T22:20:44.275 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2024-06-14T22:20:44.331 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="CentOS Stream" 2024-06-14T22:20:44.331 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="9" 2024-06-14T22:20:44.331 INFO:teuthology.orchestra.run.smithi178.stdout:ID="centos" 2024-06-14T22:20:44.332 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="rhel fedora" 2024-06-14T22:20:44.332 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="9" 2024-06-14T22:20:44.332 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el9" 2024-06-14T22:20:44.332 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-14T22:20:44.332 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2024-06-14T22:20:44.332 INFO:teuthology.orchestra.run.smithi178.stdout:LOGO="fedora-logo-icon" 2024-06-14T22:20:44.332 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-14T22:20:44.332 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://centos.org/" 2024-06-14T22:20:44.332 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-14T22:20:44.332 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-14T22:20:44.332 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-14T22:20:44.333 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2024-06-14T22:20:44.357 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-14T22:20:44.365 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-14T22:20:44.383 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-14T22:20:44.383 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2024-06-14T22:20:44.386 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2024-06-14T22:20:44.402 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-14T22:20:44.418 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-14T22:20:44.418 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2024-06-14T22:20:44.443 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2024-06-14T22:20:44.553 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-14T22:20:44.604 INFO:teuthology.run_tasks:Running task kernel... 2024-06-14T22:20:44.617 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-06-14T22:20:44.617 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-06-14T22:20:44.618 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-14T22:20:44.618 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-14T22:20:44.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-14T22:20:44.635 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-06-14T22:20:44.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-14T22:20:44.636 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-06-14T22:20:44.691 INFO:teuthology.orchestra.run.smithi178.stdout:5.14.0-437.el9.x86_64 2024-06-14T22:20:44.692 INFO:teuthology.task.kernel:Running kernel on smithi178: 5.14.0-437.el9.x86_64 2024-06-14T22:20:44.692 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum install -y kernel 2024-06-14T22:20:44.694 INFO:teuthology.orchestra.run.smithi046.stdout:5.14.0-437.el9.x86_64 2024-06-14T22:20:44.694 INFO:teuthology.task.kernel:Running kernel on smithi046: 5.14.0-437.el9.x86_64 2024-06-14T22:20:44.694 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum install -y kernel 2024-06-14T22:20:47.029 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-06-14T22:20:48.331 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 4.1 MB/s | 8.1 MB 00:01 2024-06-14T22:20:50.016 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 36 MB/s | 20 MB 00:00 2024-06-14T22:20:55.141 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.3 MB 00:01 2024-06-14T22:20:56.550 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 80 kB/s | 17 kB 00:00 2024-06-14T22:20:57.429 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 29 MB/s | 21 MB 00:00 2024-06-14T22:21:00.088 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 1.6 MB/s | 20 MB 00:11 2024-06-14T22:21:02.618 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-06-14T22:21:04.297 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.3 MB 00:00 2024-06-14T22:21:04.649 INFO:teuthology.orchestra.run.smithi178.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-14T22:21:04.649 INFO:teuthology.orchestra.run.smithi178.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-14T22:21:04.741 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:21:04.749 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:21:04.749 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-06-14T22:21:04.749 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:21:04.750 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-06-14T22:21:04.750 INFO:teuthology.orchestra.run.smithi178.stdout: kernel x86_64 5.14.0-457.el9 baseos 989 k 2024-06-14T22:21:04.750 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-06-14T22:21:04.750 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-core x86_64 5.14.0-457.el9 baseos 16 M 2024-06-14T22:21:04.750 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules x86_64 5.14.0-457.el9 baseos 35 M 2024-06-14T22:21:04.750 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules-core x86_64 5.14.0-457.el9 baseos 29 M 2024-06-14T22:21:04.750 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:21:04.750 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-06-14T22:21:04.750 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:21:04.750 INFO:teuthology.orchestra.run.smithi178.stdout:Install 4 Packages 2024-06-14T22:21:04.750 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:21:04.751 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 81 M 2024-06-14T22:21:04.751 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 124 M 2024-06-14T22:21:04.751 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-06-14T22:21:05.019 INFO:teuthology.orchestra.run.smithi178.stdout:(1/4): kernel-5.14.0-457.el9.x86_64.rpm 6.3 MB/s | 989 kB 00:00 2024-06-14T22:21:05.286 INFO:teuthology.orchestra.run.smithi178.stdout:(2/4): kernel-core-5.14.0-457.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-06-14T22:21:05.712 INFO:teuthology.orchestra.run.smithi178.stdout:(3/4): kernel-modules-5.14.0-457.el9.x86_64.rpm 41 MB/s | 35 MB 00:00 2024-06-14T22:21:05.979 INFO:teuthology.orchestra.run.smithi178.stdout:(4/4): kernel-modules-core-5.14.0-457.el9.x86_6 30 MB/s | 29 MB 00:00 2024-06-14T22:21:05.979 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:21:05.979 INFO:teuthology.orchestra.run.smithi178.stdout:Total 66 MB/s | 81 MB 00:01 2024-06-14T22:21:06.264 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 23 kB/s | 17 kB 00:00 2024-06-14T22:21:06.679 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-06-14T22:21:07.004 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-06-14T22:21:07.004 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-06-14T22:21:07.126 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 30 MB/s | 21 MB 00:00 2024-06-14T22:21:07.987 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-06-14T22:21:07.987 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-06-14T22:21:10.510 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-06-14T22:21:11.163 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-modules-core-5.14.0-457.el9.x86_64 1/4 2024-06-14T22:21:11.262 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-core-5.14.0-457.el9.x86_64 2/4 2024-06-14T22:21:12.641 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-06-14T22:21:13.908 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-core-5.14.0-457.el9.x86_64 2/4 2024-06-14T22:21:14.055 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-modules-5.14.0-457.el9.x86_64 3/4 2024-06-14T22:21:14.780 INFO:teuthology.orchestra.run.smithi046.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-14T22:21:14.780 INFO:teuthology.orchestra.run.smithi046.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-14T22:21:14.871 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:21:14.880 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:21:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: Package Architecture Version Repository Size 2024-06-14T22:21:14.880 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:21:14.880 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-06-14T22:21:14.880 INFO:teuthology.orchestra.run.smithi046.stdout: kernel x86_64 5.14.0-457.el9 baseos 989 k 2024-06-14T22:21:14.880 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-06-14T22:21:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-core x86_64 5.14.0-457.el9 baseos 16 M 2024-06-14T22:21:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules x86_64 5.14.0-457.el9 baseos 35 M 2024-06-14T22:21:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-core x86_64 5.14.0-457.el9 baseos 29 M 2024-06-14T22:21:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:21:14.881 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-06-14T22:21:14.881 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:21:14.881 INFO:teuthology.orchestra.run.smithi046.stdout:Install 4 Packages 2024-06-14T22:21:14.881 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:21:14.882 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 81 M 2024-06-14T22:21:14.882 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 124 M 2024-06-14T22:21:14.882 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-06-14T22:21:15.634 INFO:teuthology.orchestra.run.smithi046.stdout:(1/4): kernel-5.14.0-457.el9.x86_64.rpm 2.1 MB/s | 989 kB 00:00 2024-06-14T22:21:18.134 INFO:teuthology.orchestra.run.smithi046.stdout:(2/4): kernel-core-5.14.0-457.el9.x86_64.rpm 5.4 MB/s | 16 MB 00:02 2024-06-14T22:21:18.814 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-modules-5.14.0-457.el9.x86_64 3/4 2024-06-14T22:21:18.960 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-5.14.0-457.el9.x86_64 4/4 2024-06-14T22:21:20.825 INFO:teuthology.orchestra.run.smithi046.stdout:(3/4): kernel-modules-core-5.14.0-457.el9.x86_6 5.6 MB/s | 29 MB 00:05 2024-06-14T22:21:21.292 INFO:teuthology.orchestra.run.smithi046.stdout:(4/4): kernel-modules-5.14.0-457.el9.x86_64.rpm 5.7 MB/s | 35 MB 00:06 2024-06-14T22:21:21.293 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:21:21.293 INFO:teuthology.orchestra.run.smithi046.stdout:Total 13 MB/s | 81 MB 00:06 2024-06-14T22:21:22.026 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-06-14T22:21:22.453 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-06-14T22:21:22.453 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-06-14T22:21:23.456 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-06-14T22:21:23.456 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-06-14T22:21:23.707 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-modules-core-5.14.0-457.el9.x86_64 4/4 2024-06-14T22:21:26.022 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-06-14T22:21:26.671 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-modules-core-5.14.0-457.el9.x86_64 1/4 2024-06-14T22:21:26.758 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-core-5.14.0-457.el9.x86_64 2/4 2024-06-14T22:21:29.545 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-core-5.14.0-457.el9.x86_64 2/4 2024-06-14T22:21:29.684 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-modules-5.14.0-457.el9.x86_64 3/4 2024-06-14T22:21:34.789 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-5.14.0-457.el9.x86_64 3/4 2024-06-14T22:21:34.839 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-5.14.0-457.el9.x86_64 4/4 2024-06-14T22:21:39.965 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-core-5.14.0-457.el9.x86_64 4/4 2024-06-14T22:21:53.264 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-core-5.14.0-457.el9.x86_64 4/4 2024-06-14T22:21:53.304 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-modules-5.14.0-457.el9.x86_64 4/4 2024-06-14T22:21:53.601 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-5.14.0-457.el9.x86_64 4/4 2024-06-14T22:21:53.602 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-5.14.0-457.el9.x86_64 1/4 2024-06-14T22:21:53.602 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-core-5.14.0-457.el9.x86_64 2/4 2024-06-14T22:21:53.602 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-modules-5.14.0-457.el9.x86_64 3/4 2024-06-14T22:21:54.398 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-modules-core-5.14.0-457.el9.x86_64 4/4 2024-06-14T22:21:54.398 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:21:54.398 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-06-14T22:21:54.399 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-5.14.0-457.el9.x86_64 2024-06-14T22:21:54.399 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-core-5.14.0-457.el9.x86_64 2024-06-14T22:21:54.399 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules-5.14.0-457.el9.x86_64 2024-06-14T22:21:54.399 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules-core-5.14.0-457.el9.x86_64 2024-06-14T22:21:54.399 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:21:54.399 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:21:54.767 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q kernel | sort -rV | head -n 1 2024-06-14T22:21:54.848 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-5.14.0-457.el9.x86_64 2024-06-14T22:21:54.848 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-457.el9.x86_64 2024-06-14T22:21:54.848 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-457.el9.x86_64 2024-06-14T22:21:54.848 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-14T22:21:54.848 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-14T22:21:54.848 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-06-14T22:21:54.849 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi178.front.sepia.ceph.com, path=None, version=distro) 2024-06-14T22:21:54.849 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q kernel | sort -rV | head -n 1 2024-06-14T22:21:54.901 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-5.14.0-457.el9.x86_64 2024-06-14T22:21:54.902 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-457.el9.x86_64 2024-06-14T22:21:54.903 DEBUG:teuthology.orchestra.run.smithi178:> sudo rpm -qi grub2-tools 2024-06-14T22:21:54.935 INFO:teuthology.orchestra.run.smithi178.stdout:Name : grub2-tools 2024-06-14T22:21:54.936 INFO:teuthology.orchestra.run.smithi178.stdout:Epoch : 1 2024-06-14T22:21:54.936 INFO:teuthology.orchestra.run.smithi178.stdout:Version : 2.06 2024-06-14T22:21:54.936 INFO:teuthology.orchestra.run.smithi178.stdout:Release : 68.el9 2024-06-14T22:21:54.936 INFO:teuthology.orchestra.run.smithi178.stdout:Architecture: x86_64 2024-06-14T22:21:54.936 INFO:teuthology.orchestra.run.smithi178.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-14T22:21:54.936 INFO:teuthology.orchestra.run.smithi178.stdout:Group : Unspecified 2024-06-14T22:21:54.937 INFO:teuthology.orchestra.run.smithi178.stdout:Size : 8264254 2024-06-14T22:21:54.937 INFO:teuthology.orchestra.run.smithi178.stdout:License : GPLv3+ 2024-06-14T22:21:54.937 INFO:teuthology.orchestra.run.smithi178.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-14T22:21:54.937 INFO:teuthology.orchestra.run.smithi178.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-14T22:21:54.937 INFO:teuthology.orchestra.run.smithi178.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-14T22:21:54.938 INFO:teuthology.orchestra.run.smithi178.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-14T22:21:54.938 INFO:teuthology.orchestra.run.smithi178.stdout:Packager : builder@centos.org 2024-06-14T22:21:54.938 INFO:teuthology.orchestra.run.smithi178.stdout:Vendor : CentOS 2024-06-14T22:21:54.938 INFO:teuthology.orchestra.run.smithi178.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-14T22:21:54.938 INFO:teuthology.orchestra.run.smithi178.stdout:Summary : Support tools for GRUB. 2024-06-14T22:21:54.938 INFO:teuthology.orchestra.run.smithi178.stdout:Description : 2024-06-14T22:21:54.938 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:21:54.938 INFO:teuthology.orchestra.run.smithi178.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-14T22:21:54.938 INFO:teuthology.orchestra.run.smithi178.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-14T22:21:54.939 INFO:teuthology.orchestra.run.smithi178.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-14T22:21:54.939 INFO:teuthology.orchestra.run.smithi178.stdout:hardware devices. 2024-06-14T22:21:54.939 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:21:54.939 INFO:teuthology.orchestra.run.smithi178.stdout:This subpackage provides tools for support of all platforms. 2024-06-14T22:21:54.940 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-14T22:21:54.940 INFO:teuthology.task.kernel:Updating grub on smithi178 to boot 5.14.0-457.el9.x86_64 2024-06-14T22:21:54.940 DEBUG:teuthology.orchestra.run.smithi178:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-14T22:21:55.525 INFO:teuthology.orchestra.run.smithi178.stderr:Generating grub configuration file ... 2024-06-14T22:21:56.913 INFO:teuthology.orchestra.run.smithi178.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-14T22:21:56.944 INFO:teuthology.orchestra.run.smithi178.stderr:done 2024-06-14T22:21:56.946 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-06-14T22:21:56.965 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.ZOXffLc8fy 2024-06-14T22:21:56.965 DEBUG:teuthology.orchestra.run.smithi178:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ZOXffLc8fy 2024-06-14T22:21:57.039 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.ZOXffLc8fy 2024-06-14T22:21:57.180 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.ZOXffLc8fy is 6KB 2024-06-14T22:21:57.228 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.ZOXffLc8fy 2024-06-14T22:21:57.242 DEBUG:teuthology.orchestra.run.smithi178:> sudo /bin/ls /boot/loader/entries || true 2024-06-14T22:21:57.304 INFO:teuthology.orchestra.run.smithi178.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-14T22:21:57.304 INFO:teuthology.orchestra.run.smithi178.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-14T22:21:57.304 INFO:teuthology.orchestra.run.smithi178.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-14T22:21:57.304 INFO:teuthology.orchestra.run.smithi178.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-457.el9.x86_64.conf 2024-06-14T22:21:57.306 DEBUG:teuthology.orchestra.run.smithi178:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-457.el9.x86_64 2024-06-14T22:21:57.452 DEBUG:teuthology.orchestra.run.smithi178:> sudo shutdown -r now 2024-06-14T22:22:10.735 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-core-5.14.0-457.el9.x86_64 4/4 2024-06-14T22:22:10.778 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-5.14.0-457.el9.x86_64 4/4 2024-06-14T22:22:11.287 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-5.14.0-457.el9.x86_64 4/4 2024-06-14T22:22:11.287 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-5.14.0-457.el9.x86_64 1/4 2024-06-14T22:22:11.287 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-core-5.14.0-457.el9.x86_64 2/4 2024-06-14T22:22:11.287 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-modules-5.14.0-457.el9.x86_64 3/4 2024-06-14T22:22:12.119 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-modules-core-5.14.0-457.el9.x86_64 4/4 2024-06-14T22:22:12.119 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:22:12.119 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-06-14T22:22:12.119 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-5.14.0-457.el9.x86_64 2024-06-14T22:22:12.119 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-core-5.14.0-457.el9.x86_64 2024-06-14T22:22:12.119 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-5.14.0-457.el9.x86_64 2024-06-14T22:22:12.120 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-core-5.14.0-457.el9.x86_64 2024-06-14T22:22:12.120 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:22:12.120 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:22:12.476 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q kernel | sort -rV | head -n 1 2024-06-14T22:22:12.566 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-5.14.0-457.el9.x86_64 2024-06-14T22:22:12.566 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-457.el9.x86_64 2024-06-14T22:22:12.566 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-457.el9.x86_64 2024-06-14T22:22:12.566 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-14T22:22:12.566 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-14T22:22:12.566 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-06-14T22:22:12.567 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi046.front.sepia.ceph.com, path=None, version=distro) 2024-06-14T22:22:12.567 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q kernel | sort -rV | head -n 1 2024-06-14T22:22:12.621 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-5.14.0-457.el9.x86_64 2024-06-14T22:22:12.622 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-457.el9.x86_64 2024-06-14T22:22:12.622 DEBUG:teuthology.orchestra.run.smithi046:> sudo rpm -qi grub2-tools 2024-06-14T22:22:12.653 INFO:teuthology.orchestra.run.smithi046.stdout:Name : grub2-tools 2024-06-14T22:22:12.653 INFO:teuthology.orchestra.run.smithi046.stdout:Epoch : 1 2024-06-14T22:22:12.653 INFO:teuthology.orchestra.run.smithi046.stdout:Version : 2.06 2024-06-14T22:22:12.653 INFO:teuthology.orchestra.run.smithi046.stdout:Release : 68.el9 2024-06-14T22:22:12.653 INFO:teuthology.orchestra.run.smithi046.stdout:Architecture: x86_64 2024-06-14T22:22:12.654 INFO:teuthology.orchestra.run.smithi046.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-14T22:22:12.654 INFO:teuthology.orchestra.run.smithi046.stdout:Group : Unspecified 2024-06-14T22:22:12.654 INFO:teuthology.orchestra.run.smithi046.stdout:Size : 8264254 2024-06-14T22:22:12.654 INFO:teuthology.orchestra.run.smithi046.stdout:License : GPLv3+ 2024-06-14T22:22:12.654 INFO:teuthology.orchestra.run.smithi046.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-14T22:22:12.654 INFO:teuthology.orchestra.run.smithi046.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-14T22:22:12.654 INFO:teuthology.orchestra.run.smithi046.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-14T22:22:12.654 INFO:teuthology.orchestra.run.smithi046.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-14T22:22:12.654 INFO:teuthology.orchestra.run.smithi046.stdout:Packager : builder@centos.org 2024-06-14T22:22:12.654 INFO:teuthology.orchestra.run.smithi046.stdout:Vendor : CentOS 2024-06-14T22:22:12.654 INFO:teuthology.orchestra.run.smithi046.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-14T22:22:12.655 INFO:teuthology.orchestra.run.smithi046.stdout:Summary : Support tools for GRUB. 2024-06-14T22:22:12.655 INFO:teuthology.orchestra.run.smithi046.stdout:Description : 2024-06-14T22:22:12.655 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:22:12.655 INFO:teuthology.orchestra.run.smithi046.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-14T22:22:12.655 INFO:teuthology.orchestra.run.smithi046.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-14T22:22:12.655 INFO:teuthology.orchestra.run.smithi046.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-14T22:22:12.655 INFO:teuthology.orchestra.run.smithi046.stdout:hardware devices. 2024-06-14T22:22:12.655 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:22:12.655 INFO:teuthology.orchestra.run.smithi046.stdout:This subpackage provides tools for support of all platforms. 2024-06-14T22:22:12.656 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-14T22:22:12.656 INFO:teuthology.task.kernel:Updating grub on smithi046 to boot 5.14.0-457.el9.x86_64 2024-06-14T22:22:12.656 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-14T22:22:13.305 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2024-06-14T22:22:14.642 INFO:teuthology.orchestra.run.smithi046.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-14T22:22:14.670 INFO:teuthology.orchestra.run.smithi046.stderr:done 2024-06-14T22:22:14.673 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-06-14T22:22:14.688 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.kcazQjOyYL 2024-06-14T22:22:14.688 DEBUG:teuthology.orchestra.run.smithi046:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.kcazQjOyYL 2024-06-14T22:22:14.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.kcazQjOyYL 2024-06-14T22:22:14.895 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.kcazQjOyYL is 6KB 2024-06-14T22:22:14.915 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.kcazQjOyYL 2024-06-14T22:22:14.930 DEBUG:teuthology.orchestra.run.smithi046:> sudo /bin/ls /boot/loader/entries || true 2024-06-14T22:22:14.995 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-14T22:22:14.995 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-14T22:22:14.995 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-14T22:22:14.995 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-457.el9.x86_64.conf 2024-06-14T22:22:14.996 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-457.el9.x86_64 2024-06-14T22:22:15.140 DEBUG:teuthology.orchestra.run.smithi046:> sudo shutdown -r now 2024-06-14T22:22:27.467 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-06-14T22:22:27.467 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-06-14T22:22:27.468 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-14T22:22:45.156 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-06-14T22:22:45.157 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-06-14T22:22:45.157 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-14T22:23:01.919 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-06-14T22:23:10.923 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-06-14T22:23:10.924 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-14T22:23:19.838 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-06-14T22:23:28.847 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-06-14T22:23:28.848 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-14T22:23:47.231 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-06-14T22:23:59.232 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-06-14T22:23:59.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-14T22:23:59.624 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-06-14T22:24:00.063 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-06-14T22:24:00.064 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-457.el9.x86_64"... 2024-06-14T22:24:00.064 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-06-14T22:24:00.079 INFO:teuthology.orchestra.run.smithi046.stdout:5.14.0-457.el9.x86_64 2024-06-14T22:24:00.079 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-457.el9.x86_64 vs 5.14.0-457.el9.x86_64 2024-06-14T22:24:00.079 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-14T22:24:00.079 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-14T22:24:01.080 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-06-14T22:24:01.081 DEBUG:teuthology.orchestra.run.smithi046:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-14T22:24:01.177 INFO:teuthology.orchestra.run.smithi046.stdout:ttyS1 2024-06-14T22:24:01.211 DEBUG:teuthology.parallel:result is None 2024-06-14T22:24:10.927 DEBUG:teuthology.orchestra.remote:timed out 2024-06-14T22:24:22.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-06-14T22:24:22.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-14T22:24:23.296 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-06-14T22:24:23.695 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-06-14T22:24:23.695 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-457.el9.x86_64"... 2024-06-14T22:24:23.695 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-06-14T22:24:23.712 INFO:teuthology.orchestra.run.smithi178.stdout:5.14.0-457.el9.x86_64 2024-06-14T22:24:23.712 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-457.el9.x86_64 vs 5.14.0-457.el9.x86_64 2024-06-14T22:24:23.713 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-14T22:24:23.713 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-14T22:24:24.713 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-06-14T22:24:24.714 DEBUG:teuthology.orchestra.run.smithi178:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-14T22:24:24.783 INFO:teuthology.orchestra.run.smithi178.stdout:ttyS1 2024-06-14T22:24:24.821 DEBUG:teuthology.parallel:result is None 2024-06-14T22:24:24.822 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-14T22:24:25.023 INFO:teuthology.task.internal:Creating test directory... 2024-06-14T22:24:25.023 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-14T22:24:25.027 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-14T22:24:25.046 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-14T22:24:25.064 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-14T22:24:25.101 INFO:teuthology.task.internal:Creating archive directory... 2024-06-14T22:24:25.102 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-14T22:24:25.105 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-14T22:24:25.151 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-14T22:24:25.158 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-14T22:24:25.158 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-14T22:24:25.187 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-14T22:24:25.218 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-14T22:24:25.225 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-14T22:24:25.230 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-14T22:24:25.237 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-14T22:24:25.238 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-14T22:24:25.245 INFO:teuthology.task.internal:Configuring sudo... 2024-06-14T22:24:25.245 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-14T22:24:25.275 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-14T22:24:25.309 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-14T22:24:25.318 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-14T22:24:25.319 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-14T22:24:25.344 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-14T22:24:25.367 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-14T22:24:25.445 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-14T22:24:25.494 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-14T22:24:25.577 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-14T22:24:25.620 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-06-14T22:24:25.620 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-14T22:24:25.688 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-14T22:24:25.735 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-14T22:24:25.784 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-14T22:24:25.869 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-14T22:24:25.912 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-14T22:24:25.912 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-14T22:24:25.979 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2024-06-14T22:24:25.982 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2024-06-14T22:24:26.023 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-14T22:24:26.071 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-14T22:24:26.558 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-14T22:24:26.566 INFO:teuthology.task.internal:Starting timer... 2024-06-14T22:24:26.567 INFO:teuthology.run_tasks:Running task pcp... 2024-06-14T22:24:26.606 INFO:teuthology.run_tasks:Running task selinux... 2024-06-14T22:24:26.646 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-06-14T22:24:26.646 DEBUG:teuthology.orchestra.run.smithi046:> sudo service auditd rotate 2024-06-14T22:24:26.702 INFO:teuthology.orchestra.run.smithi046.stdout:Rotating logs: 2024-06-14T22:24:26.704 DEBUG:teuthology.orchestra.run.smithi178:> sudo service auditd rotate 2024-06-14T22:24:26.749 INFO:teuthology.orchestra.run.smithi178.stdout:Rotating logs: 2024-06-14T22:24:26.751 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-14T22:24:26.751 DEBUG:teuthology.orchestra.run.smithi046:> /usr/sbin/getenforce 2024-06-14T22:24:26.783 INFO:teuthology.orchestra.run.smithi046.stdout:Permissive 2024-06-14T22:24:26.783 DEBUG:teuthology.orchestra.run.smithi178:> /usr/sbin/getenforce 2024-06-14T22:24:26.829 INFO:teuthology.orchestra.run.smithi178.stdout:Permissive 2024-06-14T22:24:26.829 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi046.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi178.front.sepia.ceph.com': 'permissive'} 2024-06-14T22:24:26.829 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-06-14T22:24:26.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-14T22:24:26.859 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-06-14T22:24:26.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-14T22:24:26.898 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-14T22:24:26.898 DEBUG:teuthology.orchestra.run.smithi046:> sudo /usr/sbin/setenforce permissive 2024-06-14T22:24:26.927 DEBUG:teuthology.orchestra.run.smithi178:> sudo /usr/sbin/setenforce permissive 2024-06-14T22:24:26.968 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-14T22:24:26.978 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-14T22:24:26.990 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-14T22:24:26.991 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi046.front.sepia.ceph.com,smithi178.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-14T22:28:53.506 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi046.front.sepia.ceph.com'), Remote(name='ubuntu@smithi178.front.sepia.ceph.com')] 2024-06-14T22:28:53.508 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-06-14T22:28:53.509 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-14T22:28:53.589 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-06-14T22:28:53.675 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-06-14T22:28:53.675 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-06-14T22:28:53.676 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-14T22:28:53.745 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-06-14T22:28:53.825 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-06-14T22:28:53.825 INFO:teuthology.run_tasks:Running task clock... 2024-06-14T22:28:53.835 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-14T22:28:53.836 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-14T22:28:53.836 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-14T22:28:53.839 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-14T22:28:53.839 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-14T22:28:53.868 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-14T22:28:53.883 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-14T22:28:53.895 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-14T22:28:53.908 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: ntpd: command not found 2024-06-14T22:28:53.909 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-14T22:28:53.919 INFO:teuthology.orchestra.run.smithi046.stdout:506 Cannot talk to daemon 2024-06-14T22:28:53.933 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: ntpd: command not found 2024-06-14T22:28:53.934 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-14T22:28:53.944 INFO:teuthology.orchestra.run.smithi178.stdout:506 Cannot talk to daemon 2024-06-14T22:28:53.949 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-14T22:28:53.958 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-14T22:28:53.972 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-14T22:28:54.003 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-06-14T22:28:54.007 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-14T22:28:54.007 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-06-14T22:28:54.007 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-14T22:28:54.007 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-14T22:28:54.007 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-14T22:28:54.007 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-14T22:28:54.031 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-06-14T22:28:54.035 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-14T22:28:54.035 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-06-14T22:28:54.035 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-14T22:28:54.035 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-14T22:28:54.036 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-14T22:28:54.036 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-14T22:28:54.036 INFO:teuthology.run_tasks:Running task pexec... 2024-06-14T22:28:54.045 INFO:teuthology.task.pexec:Executing custom commands... 2024-06-14T22:28:54.045 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi046.front.sepia.ceph.com 2024-06-14T22:28:54.045 DEBUG:teuthology.orchestra.run.smithi046:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-06-14T22:28:54.046 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi178.front.sepia.ceph.com 2024-06-14T22:28:54.046 DEBUG:teuthology.orchestra.run.smithi178:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-06-14T22:28:54.667 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:09 ago on Fri 14 Jun 2024 10:27:45 PM UTC. 2024-06-14T22:28:54.672 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:16 ago on Fri 14 Jun 2024 10:27:38 PM UTC. 2024-06-14T22:28:54.795 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:28:54.795 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:28:54.809 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:28:54.809 INFO:teuthology.orchestra.run.smithi046.stdout: Package Architecture Version Repository Size 2024-06-14T22:28:54.809 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:28:54.809 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-06-14T22:28:54.809 INFO:teuthology.orchestra.run.smithi046.stdout: runc x86_64 4:1.1.12-2.el9 appstream 3.1 M 2024-06-14T22:28:54.809 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:28:54.809 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-06-14T22:28:54.809 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:28:54.809 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-06-14T22:28:54.810 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:28:54.810 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 3.1 M 2024-06-14T22:28:54.810 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 9.8 M 2024-06-14T22:28:54.810 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-06-14T22:28:54.814 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:28:54.814 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-06-14T22:28:54.814 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:28:54.814 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-06-14T22:28:54.814 INFO:teuthology.orchestra.run.smithi178.stdout: runc x86_64 4:1.1.12-2.el9 appstream 3.1 M 2024-06-14T22:28:54.814 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:28:54.814 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-06-14T22:28:54.814 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:28:54.814 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-06-14T22:28:54.815 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:28:54.815 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 3.1 M 2024-06-14T22:28:54.815 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 9.8 M 2024-06-14T22:28:54.815 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-06-14T22:28:55.700 INFO:teuthology.orchestra.run.smithi178.stdout:runc-1.1.12-2.el9.x86_64.rpm 3.7 MB/s | 3.1 MB 00:00 2024-06-14T22:28:55.700 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:28:55.700 INFO:teuthology.orchestra.run.smithi178.stdout:Total 3.5 MB/s | 3.1 MB 00:00 2024-06-14T22:28:55.736 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-06-14T22:28:55.742 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-06-14T22:28:55.742 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-06-14T22:28:55.825 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-06-14T22:28:55.825 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-06-14T22:28:56.041 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-06-14T22:28:56.135 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : runc-4:1.1.12-2.el9.x86_64 1/1 2024-06-14T22:28:56.215 INFO:teuthology.orchestra.run.smithi046.stdout:runc-1.1.12-2.el9.x86_64.rpm 2.8 MB/s | 3.1 MB 00:01 2024-06-14T22:28:56.215 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:28:56.215 INFO:teuthology.orchestra.run.smithi046.stdout:Total 2.2 MB/s | 3.1 MB 00:01 2024-06-14T22:28:56.255 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-06-14T22:28:56.262 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-06-14T22:28:56.262 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-06-14T22:28:56.351 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-06-14T22:28:56.351 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-06-14T22:28:56.587 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-06-14T22:28:56.682 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : runc-4:1.1.12-2.el9.x86_64 1/1 2024-06-14T22:28:56.710 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: runc-4:1.1.12-2.el9.x86_64 1/1 2024-06-14T22:28:56.950 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : runc-4:1.1.12-2.el9.x86_64 1/1 2024-06-14T22:28:56.950 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:28:56.950 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-06-14T22:28:56.951 INFO:teuthology.orchestra.run.smithi178.stdout: runc-4:1.1.12-2.el9.x86_64 2024-06-14T22:28:56.951 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:28:56.951 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:28:57.209 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: runc-4:1.1.12-2.el9.x86_64 1/1 2024-06-14T22:28:57.210 DEBUG:teuthology.parallel:result is None 2024-06-14T22:28:57.503 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : runc-4:1.1.12-2.el9.x86_64 1/1 2024-06-14T22:28:57.503 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:28:57.503 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-06-14T22:28:57.503 INFO:teuthology.orchestra.run.smithi046.stdout: runc-4:1.1.12-2.el9.x86_64 2024-06-14T22:28:57.503 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:28:57.503 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:28:57.768 DEBUG:teuthology.parallel:result is None 2024-06-14T22:28:57.768 INFO:teuthology.run_tasks:Running task install... 2024-06-14T22:28:57.776 DEBUG:teuthology.task.install:project ceph 2024-06-14T22:28:57.776 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '6f4b09560ed399c3f139402a02f389c2fc8bb2cc'}} 2024-06-14T22:28:57.776 DEBUG:teuthology.task.install:config {'branch': 'quincy', 'exclude_packages': ['ceph-volume'], 'flavor': 'default', 'sha1': '6f4b09560ed399c3f139402a02f389c2fc8bb2cc'} 2024-06-14T22:28:57.776 INFO:teuthology.task.install:Using flavor: default 2024-06-14T22:28:57.782 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-14T22:28:57.782 INFO:teuthology.task.install:extra packages: [] 2024-06-14T22:28:57.782 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6f4b09560ed399c3f139402a02f389c2fc8bb2cc', 'tag': None, 'wait_for_package': False} 2024-06-14T22:28:57.782 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-14T22:28:57.782 INFO:teuthology.packaging:ref: None 2024-06-14T22:28:57.783 INFO:teuthology.packaging:tag: None 2024-06-14T22:28:57.783 INFO:teuthology.packaging:branch: quincy 2024-06-14T22:28:57.783 INFO:teuthology.packaging:sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-14T22:28:57.783 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-06-14T22:28:57.785 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': 'quincy', 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6f4b09560ed399c3f139402a02f389c2fc8bb2cc', 'tag': None, 'wait_for_package': False} 2024-06-14T22:28:57.785 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-14T22:28:57.785 INFO:teuthology.packaging:ref: None 2024-06-14T22:28:57.785 INFO:teuthology.packaging:tag: None 2024-06-14T22:28:57.785 INFO:teuthology.packaging:branch: quincy 2024-06-14T22:28:57.785 INFO:teuthology.packaging:sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-14T22:28:57.785 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-06-14T22:28:57.999 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/e47ff4b87f747658fc7376cc7065fc56801eb37b/centos/9/flavors/default/ 2024-06-14T22:28:58.000 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1008.ge47ff4b8 2024-06-14T22:28:58.015 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/quincy/e47ff4b87f747658fc7376cc7065fc56801eb37b/centos/9/flavors/default/ 2024-06-14T22:28:58.015 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1008.ge47ff4b8 2024-06-14T22:28:58.109 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/e47ff4b87f747658fc7376cc7065fc56801eb37b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/e47ff4b87f747658fc7376cc7065fc56801eb37b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/e47ff4b87f747658fc7376cc7065fc56801eb37b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-14T22:28:58.109 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-14T22:28:58.109 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-14T22:28:58.122 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/quincy/e47ff4b87f747658fc7376cc7065fc56801eb37b/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/e47ff4b87f747658fc7376cc7065fc56801eb37b/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/quincy/e47ff4b87f747658fc7376cc7065fc56801eb37b/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-14T22:28:58.122 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-06-14T22:28:58.122 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-14T22:28:58.136 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-06-14T22:28:58.136 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-14T22:28:58.137 INFO:teuthology.packaging:ref: None 2024-06-14T22:28:58.137 INFO:teuthology.packaging:tag: None 2024-06-14T22:28:58.137 INFO:teuthology.packaging:branch: quincy 2024-06-14T22:28:58.137 INFO:teuthology.packaging:sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-14T22:28:58.137 DEBUG:teuthology.orchestra.run.smithi178:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-14T22:28:58.150 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-06-14T22:28:58.151 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-14T22:28:58.151 INFO:teuthology.packaging:ref: None 2024-06-14T22:28:58.151 INFO:teuthology.packaging:tag: None 2024-06-14T22:28:58.151 INFO:teuthology.packaging:branch: quincy 2024-06-14T22:28:58.151 INFO:teuthology.packaging:sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-14T22:28:58.151 DEBUG:teuthology.orchestra.run.smithi046:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/quincy/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-14T22:28:58.204 DEBUG:teuthology.orchestra.run.smithi178:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-14T22:28:58.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-14T22:28:58.281 DEBUG:teuthology.orchestra.run.smithi178:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-14T22:28:58.297 DEBUG:teuthology.orchestra.run.smithi046:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-14T22:28:58.347 INFO:teuthology.orchestra.run.smithi178.stdout:check_obsoletes = 1 2024-06-14T22:28:58.348 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum clean all 2024-06-14T22:28:58.363 INFO:teuthology.orchestra.run.smithi046.stdout:check_obsoletes = 1 2024-06-14T22:28:58.364 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum clean all 2024-06-14T22:28:58.577 INFO:teuthology.orchestra.run.smithi178.stdout:45 files removed 2024-06-14T22:28:58.604 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-radosgw 2024-06-14T22:28:58.604 INFO:teuthology.orchestra.run.smithi046.stdout:45 files removed 2024-06-14T22:28:58.632 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-radosgw 2024-06-14T22:28:59.230 INFO:teuthology.orchestra.run.smithi178.stdout:ceph packages for x86_64 221 kB/s | 77 kB 00:00 2024-06-14T22:28:59.349 INFO:teuthology.orchestra.run.smithi046.stdout:ceph packages for x86_64 182 kB/s | 77 kB 00:00 2024-06-14T22:28:59.656 INFO:teuthology.orchestra.run.smithi178.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-06-14T22:28:59.733 INFO:teuthology.orchestra.run.smithi046.stdout:ceph noarch packages 32 kB/s | 11 kB 00:00 2024-06-14T22:29:00.048 INFO:teuthology.orchestra.run.smithi178.stdout:ceph source packages 5.1 kB/s | 1.8 kB 00:00 2024-06-14T22:29:00.158 INFO:teuthology.orchestra.run.smithi046.stdout:ceph source packages 4.7 kB/s | 1.8 kB 00:00 2024-06-14T22:29:01.433 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.1 MB 00:01 2024-06-14T22:29:01.534 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.1 MB 00:01 2024-06-14T22:29:03.193 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 35 MB/s | 20 MB 00:00 2024-06-14T22:29:03.516 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-06-14T22:29:07.519 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.3 MB 00:00 2024-06-14T22:29:08.742 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.3 MB 00:01 2024-06-14T22:29:08.969 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 97 kB/s | 17 kB 00:00 2024-06-14T22:29:10.191 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-06-14T22:29:11.142 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 14 kB/s | 17 kB 00:01 2024-06-14T22:29:11.971 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 31 MB/s | 21 MB 00:00 2024-06-14T22:29:15.705 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-06-14T22:29:17.185 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-06-14T22:29:18.222 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:29:18.224 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:29:18.225 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-06-14T22:29:18.225 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:29:18.225 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-06-14T22:29:18.225 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 14 M 2024-06-14T22:29:18.225 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrading: 2024-06-14T22:29:18.225 INFO:teuthology.orchestra.run.smithi046.stdout: librados2 x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 3.1 M 2024-06-14T22:29:18.225 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1 x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 2.9 M 2024-06-14T22:29:18.225 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-06-14T22:29:18.225 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 5.0 M 2024-06-14T22:29:18.226 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 22 M 2024-06-14T22:29:18.226 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 25 k 2024-06-14T22:29:18.226 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-14T22:29:18.226 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2 x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 638 k 2024-06-14T22:29:18.226 INFO:teuthology.orchestra.run.smithi046.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-14T22:29:18.226 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-14T22:29:18.226 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-14T22:29:18.226 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 457 k 2024-06-14T22:29:18.226 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-14T22:29:18.226 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2 x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 4.9 M 2024-06-14T22:29:18.226 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-14T22:29:18.227 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-14T22:29:18.227 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 45 k 2024-06-14T22:29:18.227 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 118 k 2024-06-14T22:29:18.227 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 156 k 2024-06-14T22:29:18.227 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-14T22:29:18.227 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 320 k 2024-06-14T22:29:18.227 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 292 k 2024-06-14T22:29:18.227 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 100 k 2024-06-14T22:29:18.227 INFO:teuthology.orchestra.run.smithi046.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-14T22:29:18.227 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:29:18.227 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-06-14T22:29:18.228 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:29:18.228 INFO:teuthology.orchestra.run.smithi046.stdout:Install 22 Packages 2024-06-14T22:29:18.228 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrade 2 Packages 2024-06-14T22:29:18.228 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:29:18.228 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 58 M 2024-06-14T22:29:18.228 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-06-14T22:29:19.022 INFO:teuthology.orchestra.run.smithi046.stdout:(1/24): ceph-base-17.2.7-1008.ge47ff4b8.el9.x86 7.4 MB/s | 5.0 MB 00:00 2024-06-14T22:29:19.073 INFO:teuthology.orchestra.run.smithi046.stdout:(2/24): ceph-selinux-17.2.7-1008.ge47ff4b8.el9. 503 kB/s | 25 kB 00:00 2024-06-14T22:29:19.156 INFO:teuthology.orchestra.run.smithi046.stdout:(3/24): libcephfs2-17.2.7-1008.ge47ff4b8.el9.x8 7.5 MB/s | 638 kB 00:00 2024-06-14T22:29:19.231 INFO:teuthology.orchestra.run.smithi046.stdout:(4/24): libradosstriper1-17.2.7-1008.ge47ff4b8. 6.0 MB/s | 457 kB 00:00 2024-06-14T22:29:19.643 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:29:19.644 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:29:19.645 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-06-14T22:29:19.645 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:29:19.645 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-06-14T22:29:19.645 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 14 M 2024-06-14T22:29:19.645 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrading: 2024-06-14T22:29:19.645 INFO:teuthology.orchestra.run.smithi178.stdout: librados2 x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 3.1 M 2024-06-14T22:29:19.645 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1 x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 2.9 M 2024-06-14T22:29:19.645 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-06-14T22:29:19.646 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 5.0 M 2024-06-14T22:29:19.646 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 22 M 2024-06-14T22:29:19.646 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 25 k 2024-06-14T22:29:19.646 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-14T22:29:19.646 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2 x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 638 k 2024-06-14T22:29:19.646 INFO:teuthology.orchestra.run.smithi178.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-14T22:29:19.646 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-14T22:29:19.646 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-14T22:29:19.646 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1 x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 457 k 2024-06-14T22:29:19.646 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-14T22:29:19.646 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2 x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 4.9 M 2024-06-14T22:29:19.647 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-14T22:29:19.647 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-14T22:29:19.647 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 45 k 2024-06-14T22:29:19.647 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 118 k 2024-06-14T22:29:19.647 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 156 k 2024-06-14T22:29:19.647 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-14T22:29:19.647 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 320 k 2024-06-14T22:29:19.647 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 292 k 2024-06-14T22:29:19.647 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 100 k 2024-06-14T22:29:19.647 INFO:teuthology.orchestra.run.smithi178.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-14T22:29:19.647 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:29:19.647 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-06-14T22:29:19.647 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:29:19.648 INFO:teuthology.orchestra.run.smithi178.stdout:Install 22 Packages 2024-06-14T22:29:19.648 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrade 2 Packages 2024-06-14T22:29:19.648 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:29:19.648 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 58 M 2024-06-14T22:29:19.648 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-06-14T22:29:19.782 INFO:teuthology.orchestra.run.smithi046.stdout:(5/24): librgw2-17.2.7-1008.ge47ff4b8.el9.x86_6 8.9 MB/s | 4.9 MB 00:00 2024-06-14T22:29:19.849 INFO:teuthology.orchestra.run.smithi046.stdout:(6/24): python3-ceph-argparse-17.2.7-1008.ge47f 678 kB/s | 45 kB 00:00 2024-06-14T22:29:20.074 INFO:teuthology.orchestra.run.smithi046.stdout:(7/24): ceph-common-17.2.7-1008.ge47ff4b8.el9.x 13 MB/s | 22 MB 00:01 2024-06-14T22:29:20.099 INFO:teuthology.orchestra.run.smithi046.stdout:(8/24): python3-ceph-common-17.2.7-1008.ge47ff4 470 kB/s | 118 kB 00:00 2024-06-14T22:29:20.141 INFO:teuthology.orchestra.run.smithi046.stdout:(9/24): python3-cephfs-17.2.7-1008.ge47ff4b8.el 2.3 MB/s | 156 kB 00:00 2024-06-14T22:29:20.166 INFO:teuthology.orchestra.run.smithi046.stdout:(10/24): python3-rados-17.2.7-1008.ge47ff4b8.el 4.7 MB/s | 320 kB 00:00 2024-06-14T22:29:20.217 INFO:teuthology.orchestra.run.smithi046.stdout:(11/24): python3-rbd-17.2.7-1008.ge47ff4b8.el9. 3.8 MB/s | 292 kB 00:00 2024-06-14T22:29:20.242 INFO:teuthology.orchestra.run.smithi046.stdout:(12/24): python3-rgw-17.2.7-1008.ge47ff4b8.el9. 1.3 MB/s | 100 kB 00:00 2024-06-14T22:29:20.386 INFO:teuthology.orchestra.run.smithi178.stdout:(1/24): ceph-base-17.2.7-1008.ge47ff4b8.el9.x86 8.0 MB/s | 5.0 MB 00:00 2024-06-14T22:29:20.419 INFO:teuthology.orchestra.run.smithi178.stdout:(2/24): ceph-selinux-17.2.7-1008.ge47ff4b8.el9. 758 kB/s | 25 kB 00:00 2024-06-14T22:29:20.486 INFO:teuthology.orchestra.run.smithi178.stdout:(3/24): libcephfs2-17.2.7-1008.ge47ff4b8.el9.x8 9.3 MB/s | 638 kB 00:00 2024-06-14T22:29:20.554 INFO:teuthology.orchestra.run.smithi178.stdout:(4/24): libradosstriper1-17.2.7-1008.ge47ff4b8. 6.7 MB/s | 457 kB 00:00 2024-06-14T22:29:20.692 INFO:teuthology.orchestra.run.smithi046.stdout:(13/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 101 kB/s | 45 kB 00:00 2024-06-14T22:29:20.784 INFO:teuthology.orchestra.run.smithi046.stdout:(14/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 282 kB/s | 160 kB 00:00 2024-06-14T22:29:21.009 INFO:teuthology.orchestra.run.smithi046.stdout:(15/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.0 MB/s | 662 kB 00:00 2024-06-14T22:29:21.035 INFO:teuthology.orchestra.run.smithi046.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.1 MB/s | 292 kB 00:00 2024-06-14T22:29:21.037 INFO:teuthology.orchestra.run.smithi178.stdout:(5/24): librgw2-17.2.7-1008.ge47ff4b8.el9.x86_6 10 MB/s | 4.9 MB 00:00 2024-06-14T22:29:21.104 INFO:teuthology.orchestra.run.smithi178.stdout:(6/24): python3-ceph-argparse-17.2.7-1008.ge47f 680 kB/s | 45 kB 00:00 2024-06-14T22:29:21.110 INFO:teuthology.orchestra.run.smithi046.stdout:(17/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 4.0 MB/s | 309 kB 00:00 2024-06-14T22:29:21.135 INFO:teuthology.orchestra.run.smithi046.stdout:(18/24): python3-prettytable-0.7.2-27.el9.noarc 338 kB/s | 42 kB 00:00 2024-06-14T22:29:21.161 INFO:teuthology.orchestra.run.smithi046.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 962 kB/s | 49 kB 00:00 2024-06-14T22:29:21.171 INFO:teuthology.orchestra.run.smithi178.stdout:(7/24): python3-ceph-common-17.2.7-1008.ge47ff4 1.7 MB/s | 118 kB 00:00 2024-06-14T22:29:21.228 INFO:teuthology.orchestra.run.smithi046.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 732 kB/s | 67 kB 00:00 2024-06-14T22:29:21.246 INFO:teuthology.orchestra.run.smithi178.stdout:(8/24): python3-cephfs-17.2.7-1008.ge47ff4b8.el 2.0 MB/s | 156 kB 00:00 2024-06-14T22:29:21.261 INFO:teuthology.orchestra.run.smithi046.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 22 MB/s | 1.6 MB 00:00 2024-06-14T22:29:21.338 INFO:teuthology.orchestra.run.smithi178.stdout:(9/24): python3-rados-17.2.7-1008.ge47ff4b8.el9 3.4 MB/s | 320 kB 00:00 2024-06-14T22:29:21.430 INFO:teuthology.orchestra.run.smithi178.stdout:(10/24): python3-rbd-17.2.7-1008.ge47ff4b8.el9. 3.1 MB/s | 292 kB 00:00 2024-06-14T22:29:21.497 INFO:teuthology.orchestra.run.smithi178.stdout:(11/24): python3-rgw-17.2.7-1008.ge47ff4b8.el9. 1.5 MB/s | 100 kB 00:00 2024-06-14T22:29:21.528 INFO:teuthology.orchestra.run.smithi046.stdout:(22/24): ceph-radosgw-17.2.7-1008.ge47ff4b8.el9 4.4 MB/s | 14 MB 00:03 2024-06-14T22:29:21.845 INFO:teuthology.orchestra.run.smithi046.stdout:(23/24): librados2-17.2.7-1008.ge47ff4b8.el9.x8 5.0 MB/s | 3.1 MB 00:00 2024-06-14T22:29:21.920 INFO:teuthology.orchestra.run.smithi046.stdout:(24/24): librbd1-17.2.7-1008.ge47ff4b8.el9.x86_ 4.5 MB/s | 2.9 MB 00:00 2024-06-14T22:29:21.927 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:29:21.927 INFO:teuthology.orchestra.run.smithi046.stdout:Total 16 MB/s | 58 MB 00:03 2024-06-14T22:29:21.993 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-06-14T22:29:22.031 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-06-14T22:29:22.032 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-06-14T22:29:22.256 INFO:teuthology.orchestra.run.smithi178.stdout:(12/24): ceph-common-17.2.7-1008.ge47ff4b8.el9. 9.0 MB/s | 22 MB 00:02 2024-06-14T22:29:22.281 INFO:teuthology.orchestra.run.smithi178.stdout:(13/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 204 kB/s | 160 kB 00:00 2024-06-14T22:29:22.373 INFO:teuthology.orchestra.run.smithi178.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 389 kB/s | 45 kB 00:00 2024-06-14T22:29:22.565 INFO:teuthology.orchestra.run.smithi178.stdout:(15/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 662 kB 00:00 2024-06-14T22:29:22.597 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-06-14T22:29:22.598 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-06-14T22:29:22.615 INFO:teuthology.orchestra.run.smithi178.stdout:(16/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.2 MB/s | 292 kB 00:00 2024-06-14T22:29:22.666 INFO:teuthology.orchestra.run.smithi178.stdout:(17/24): python3-prettytable-0.7.2-27.el9.noarc 423 kB/s | 42 kB 00:00 2024-06-14T22:29:22.724 INFO:teuthology.orchestra.run.smithi178.stdout:(18/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.8 MB/s | 309 kB 00:00 2024-06-14T22:29:22.750 INFO:teuthology.orchestra.run.smithi178.stdout:(19/24): libunwind-1.6.2-1.el9.x86_64.rpm 2.9 MB/s | 67 kB 00:00 2024-06-14T22:29:22.775 INFO:teuthology.orchestra.run.smithi178.stdout:(20/24): liboath-2.6.7-2.el9.x86_64.rpm 456 kB/s | 49 kB 00:00 2024-06-14T22:29:22.825 INFO:teuthology.orchestra.run.smithi178.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 21 MB/s | 1.6 MB 00:00 2024-06-14T22:29:23.092 INFO:teuthology.orchestra.run.smithi178.stdout:(22/24): librados2-17.2.7-1008.ge47ff4b8.el9.x8 9.7 MB/s | 3.1 MB 00:00 2024-06-14T22:29:23.184 INFO:teuthology.orchestra.run.smithi178.stdout:(23/24): librbd1-17.2.7-1008.ge47ff4b8.el9.x86_ 8.2 MB/s | 2.9 MB 00:00 2024-06-14T22:29:23.509 INFO:teuthology.orchestra.run.smithi178.stdout:(24/24): ceph-radosgw-17.2.7-1008.ge47ff4b8.el9 3.8 MB/s | 14 MB 00:03 2024-06-14T22:29:23.515 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:29:23.515 INFO:teuthology.orchestra.run.smithi178.stdout:Total 15 MB/s | 58 MB 00:03 2024-06-14T22:29:23.579 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-06-14T22:29:23.612 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-06-14T22:29:23.612 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-06-14T22:29:23.813 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-06-14T22:29:24.163 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-06-14T22:29:24.163 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-06-14T22:29:24.217 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-06-14T22:29:24.281 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librados2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2/26 2024-06-14T22:29:24.375 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2/26 2024-06-14T22:29:24.455 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rados-2:17.2.7-1008.ge47ff4b8.el9.x86_64 3/26 2024-06-14T22:29:24.482 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 4/26 2024-06-14T22:29:24.561 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephfs2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 4/26 2024-06-14T22:29:24.610 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-06-14T22:29:24.637 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-06-14T22:29:25.094 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-06-14T22:29:25.119 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librgw2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 8/26 2024-06-14T22:29:25.207 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librgw2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 8/26 2024-06-14T22:29:25.235 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-06-14T22:29:25.279 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-argparse-2:17.2.7-1008.ge47ff4b8.el9. 10/26 2024-06-14T22:29:25.316 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cephfs-2:17.2.7-1008.ge47ff4b8.el9.x86_64 11/26 2024-06-14T22:29:25.356 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-06-14T22:29:25.379 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rgw-2:17.2.7-1008.ge47ff4b8.el9.x86_64 12/26 2024-06-14T22:29:25.405 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libradosstriper1-2:17.2.7-1008.ge47ff4b8.el9.x86_6 13/26 2024-06-14T22:29:25.487 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1008.ge47ff4b8.el9.x86_6 13/26 2024-06-14T22:29:25.529 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-06-14T22:29:25.563 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-06-14T22:29:25.596 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-06-14T22:29:25.738 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-06-14T22:29:25.802 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librados2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2/26 2024-06-14T22:29:25.901 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2/26 2024-06-14T22:29:25.905 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-06-14T22:29:25.950 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librbd1-2:17.2.7-1008.ge47ff4b8.el9.x86_64 18/26 2024-06-14T22:29:25.982 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rados-2:17.2.7-1008.ge47ff4b8.el9.x86_64 3/26 2024-06-14T22:29:26.004 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 4/26 2024-06-14T22:29:26.025 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:17.2.7-1008.ge47ff4b8.el9.x86_64 18/26 2024-06-14T22:29:26.100 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rbd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 19/26 2024-06-14T22:29:26.100 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 4/26 2024-06-14T22:29:26.128 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-common-2:17.2.7-1008.ge47ff4b8.el9.x8 20/26 2024-06-14T22:29:26.139 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-06-14T22:29:26.176 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-06-14T22:29:26.609 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-06-14T22:29:26.633 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librgw2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 8/26 2024-06-14T22:29:26.712 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librgw2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 8/26 2024-06-14T22:29:26.740 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-06-14T22:29:26.783 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-argparse-2:17.2.7-1008.ge47ff4b8.el9. 10/26 2024-06-14T22:29:26.821 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cephfs-2:17.2.7-1008.ge47ff4b8.el9.x86_64 11/26 2024-06-14T22:29:26.906 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rgw-2:17.2.7-1008.ge47ff4b8.el9.x86_64 12/26 2024-06-14T22:29:26.935 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libradosstriper1-2:17.2.7-1008.ge47ff4b8.el9.x86_6 13/26 2024-06-14T22:29:27.001 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1008.ge47ff4b8.el9.x86_6 13/26 2024-06-14T22:29:27.042 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-06-14T22:29:27.085 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-06-14T22:29:27.118 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-06-14T22:29:27.419 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-06-14T22:29:27.456 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librbd1-2:17.2.7-1008.ge47ff4b8.el9.x86_64 18/26 2024-06-14T22:29:27.549 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:17.2.7-1008.ge47ff4b8.el9.x86_64 18/26 2024-06-14T22:29:27.620 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rbd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 19/26 2024-06-14T22:29:27.650 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-common-2:17.2.7-1008.ge47ff4b8.el9.x8 20/26 2024-06-14T22:29:28.546 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:17.2.7-1008.ge47ff4b8.el9.x86_64 21/26 2024-06-14T22:29:28.583 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-common-2:17.2.7-1008.ge47ff4b8.el9.x86_64 21/26 2024-06-14T22:29:29.157 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:17.2.7-1008.ge47ff4b8.el9.x86_64 21/26 2024-06-14T22:29:29.195 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-base-2:17.2.7-1008.ge47ff4b8.el9.x86_64 22/26 2024-06-14T22:29:29.294 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-base-2:17.2.7-1008.ge47ff4b8.el9.x86_64 22/26 2024-06-14T22:29:29.294 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-14T22:29:29.295 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-14T22:29:29.295 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:29:29.329 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-selinux-2:17.2.7-1008.ge47ff4b8.el9.x86_64 23/26 2024-06-14T22:29:30.123 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:17.2.7-1008.ge47ff4b8.el9.x86_64 21/26 2024-06-14T22:29:30.166 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-common-2:17.2.7-1008.ge47ff4b8.el9.x86_64 21/26 2024-06-14T22:29:30.703 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:17.2.7-1008.ge47ff4b8.el9.x86_64 21/26 2024-06-14T22:29:30.745 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-base-2:17.2.7-1008.ge47ff4b8.el9.x86_64 22/26 2024-06-14T22:29:30.839 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-base-2:17.2.7-1008.ge47ff4b8.el9.x86_64 22/26 2024-06-14T22:29:30.839 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-14T22:29:30.839 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-14T22:29:30.840 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:29:30.871 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-selinux-2:17.2.7-1008.ge47ff4b8.el9.x86_64 23/26 2024-06-14T22:29:39.346 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1008.ge47ff4b8.el9.x86_64 23/26 2024-06-14T22:29:39.346 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /sys 2024-06-14T22:29:39.347 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /proc 2024-06-14T22:29:39.347 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /mnt 2024-06-14T22:29:39.347 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /var/tmp 2024-06-14T22:29:39.347 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /home 2024-06-14T22:29:39.347 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /root 2024-06-14T22:29:39.347 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /tmp 2024-06-14T22:29:39.347 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:29:40.407 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1008.ge47ff4b8.el9.x86_64 23/26 2024-06-14T22:29:40.408 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /sys 2024-06-14T22:29:40.408 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /proc 2024-06-14T22:29:40.408 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /mnt 2024-06-14T22:29:40.408 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /var/tmp 2024-06-14T22:29:40.408 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /home 2024-06-14T22:29:40.408 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /root 2024-06-14T22:29:40.408 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /tmp 2024-06-14T22:29:40.408 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:29:40.682 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-radosgw-2:17.2.7-1008.ge47ff4b8.el9.x86_64 24/26 2024-06-14T22:29:40.745 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1008.ge47ff4b8.el9.x86_64 24/26 2024-06-14T22:29:40.745 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-14T22:29:40.745 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-14T22:29:40.745 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-14T22:29:40.745 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-14T22:29:40.746 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:29:40.747 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-06-14T22:29:40.843 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-06-14T22:29:40.843 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-06-14T22:29:41.663 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-radosgw-2:17.2.7-1008.ge47ff4b8.el9.x86_64 24/26 2024-06-14T22:29:41.722 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1008.ge47ff4b8.el9.x86_64 24/26 2024-06-14T22:29:41.722 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-14T22:29:41.722 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-14T22:29:41.722 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-14T22:29:41.722 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-14T22:29:41.722 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:29:41.724 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-06-14T22:29:41.816 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-06-14T22:29:41.816 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-06-14T22:29:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-06-14T22:29:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-base-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/26 2024-06-14T22:29:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-common-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2/26 2024-06-14T22:29:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-radosgw-2:17.2.7-1008.ge47ff4b8.el9.x86_64 3/26 2024-06-14T22:29:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-selinux-2:17.2.7-1008.ge47ff4b8.el9.x86_64 4/26 2024-06-14T22:29:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 5/26 2024-06-14T22:29:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libradosstriper1-2:17.2.7-1008.ge47ff4b8.el9.x86_6 6/26 2024-06-14T22:29:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librgw2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 7/26 2024-06-14T22:29:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1008.ge47ff4b8.el9. 8/26 2024-06-14T22:29:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-common-2:17.2.7-1008.ge47ff4b8.el9.x8 9/26 2024-06-14T22:29:41.983 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cephfs-2:17.2.7-1008.ge47ff4b8.el9.x86_64 10/26 2024-06-14T22:29:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rados-2:17.2.7-1008.ge47ff4b8.el9.x86_64 11/26 2024-06-14T22:29:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rbd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 12/26 2024-06-14T22:29:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rgw-2:17.2.7-1008.ge47ff4b8.el9.x86_64 13/26 2024-06-14T22:29:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-06-14T22:29:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-06-14T22:29:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-06-14T22:29:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-06-14T22:29:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-06-14T22:29:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-06-14T22:29:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-06-14T22:29:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-06-14T22:29:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-06-14T22:29:41.984 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 23/26 2024-06-14T22:29:41.985 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-06-14T22:29:41.985 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:17.2.7-1008.ge47ff4b8.el9.x86_64 25/26 2024-06-14T22:29:42.846 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-06-14T22:29:42.846 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:29:42.847 INFO:teuthology.orchestra.run.smithi046.stdout:Upgraded: 2024-06-14T22:29:42.847 INFO:teuthology.orchestra.run.smithi046.stdout: librados2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:42.847 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:42.847 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-06-14T22:29:42.847 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:42.847 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:42.847 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:42.847 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:42.847 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-14T22:29:42.847 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:42.847 INFO:teuthology.orchestra.run.smithi046.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-14T22:29:42.847 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-14T22:29:42.847 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-14T22:29:42.848 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:42.848 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-14T22:29:42.848 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:42.848 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-14T22:29:42.848 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-14T22:29:42.848 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:42.848 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:42.848 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:42.848 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-14T22:29:42.848 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:42.848 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:42.848 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:42.848 INFO:teuthology.orchestra.run.smithi046.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-14T22:29:42.849 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:29:42.849 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:29:43.067 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-06-14T22:29:43.068 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-base-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/26 2024-06-14T22:29:43.068 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-common-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2/26 2024-06-14T22:29:43.068 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-radosgw-2:17.2.7-1008.ge47ff4b8.el9.x86_64 3/26 2024-06-14T22:29:43.068 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-selinux-2:17.2.7-1008.ge47ff4b8.el9.x86_64 4/26 2024-06-14T22:29:43.068 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 5/26 2024-06-14T22:29:43.068 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libradosstriper1-2:17.2.7-1008.ge47ff4b8.el9.x86_6 6/26 2024-06-14T22:29:43.068 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librgw2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 7/26 2024-06-14T22:29:43.068 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1008.ge47ff4b8.el9. 8/26 2024-06-14T22:29:43.068 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-common-2:17.2.7-1008.ge47ff4b8.el9.x8 9/26 2024-06-14T22:29:43.068 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cephfs-2:17.2.7-1008.ge47ff4b8.el9.x86_64 10/26 2024-06-14T22:29:43.068 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rados-2:17.2.7-1008.ge47ff4b8.el9.x86_64 11/26 2024-06-14T22:29:43.068 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rbd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 12/26 2024-06-14T22:29:43.068 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rgw-2:17.2.7-1008.ge47ff4b8.el9.x86_64 13/26 2024-06-14T22:29:43.069 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-06-14T22:29:43.069 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-06-14T22:29:43.069 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-06-14T22:29:43.069 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-06-14T22:29:43.069 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-06-14T22:29:43.069 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-06-14T22:29:43.069 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-06-14T22:29:43.069 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-06-14T22:29:43.069 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-06-14T22:29:43.069 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 23/26 2024-06-14T22:29:43.069 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-06-14T22:29:43.069 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:17.2.7-1008.ge47ff4b8.el9.x86_64 25/26 2024-06-14T22:29:43.229 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-test 2024-06-14T22:29:43.816 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-06-14T22:29:43.816 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:29:43.816 INFO:teuthology.orchestra.run.smithi178.stdout:Upgraded: 2024-06-14T22:29:43.816 INFO:teuthology.orchestra.run.smithi178.stdout: librados2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:43.816 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:43.816 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-06-14T22:29:43.816 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:43.817 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:43.817 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:43.817 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:43.817 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-14T22:29:43.817 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:43.817 INFO:teuthology.orchestra.run.smithi178.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-14T22:29:43.817 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-14T22:29:43.817 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-14T22:29:43.817 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:43.817 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-14T22:29:43.817 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:43.817 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-14T22:29:43.817 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-14T22:29:43.818 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:43.818 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:43.818 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:43.818 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-14T22:29:43.818 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:43.818 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:43.818 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:43.818 INFO:teuthology.orchestra.run.smithi178.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-14T22:29:43.818 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:29:43.818 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:29:43.827 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:28 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:29:43.957 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:29:43.958 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:29:43.958 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-06-14T22:29:43.958 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:29:43.958 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-06-14T22:29:43.958 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 48 M 2024-06-14T22:29:43.958 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-06-14T22:29:43.958 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 163 k 2024-06-14T22:29:43.959 INFO:teuthology.orchestra.run.smithi046.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-06-14T22:29:43.959 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-14T22:29:43.959 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:29:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-06-14T22:29:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:29:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:Install 4 Packages 2024-06-14T22:29:43.959 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:29:43.959 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 48 M 2024-06-14T22:29:43.960 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 192 M 2024-06-14T22:29:43.960 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-06-14T22:29:44.157 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-test 2024-06-14T22:29:44.239 INFO:teuthology.orchestra.run.smithi046.stdout:(1/4): libcephsqlite-17.2.7-1008.ge47ff4b8.el9. 710 kB/s | 163 kB 00:00 2024-06-14T22:29:44.614 INFO:teuthology.orchestra.run.smithi046.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 170 kB/s | 64 kB 00:00 2024-06-14T22:29:44.664 INFO:teuthology.orchestra.run.smithi046.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 464 kB/s | 304 kB 00:00 2024-06-14T22:29:44.743 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:27 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:29:44.868 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:29:44.869 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:29:44.869 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-06-14T22:29:44.869 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:29:44.869 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-06-14T22:29:44.870 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 48 M 2024-06-14T22:29:44.870 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-06-14T22:29:44.870 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 163 k 2024-06-14T22:29:44.870 INFO:teuthology.orchestra.run.smithi178.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-06-14T22:29:44.870 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-14T22:29:44.870 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:29:44.870 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-06-14T22:29:44.870 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:29:44.870 INFO:teuthology.orchestra.run.smithi178.stdout:Install 4 Packages 2024-06-14T22:29:44.870 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:29:44.871 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 48 M 2024-06-14T22:29:44.871 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 192 M 2024-06-14T22:29:44.871 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-06-14T22:29:45.289 INFO:teuthology.orchestra.run.smithi178.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.2 MB/s | 304 kB 00:00 2024-06-14T22:29:45.339 INFO:teuthology.orchestra.run.smithi178.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-14T22:29:45.406 INFO:teuthology.orchestra.run.smithi178.stdout:(3/4): libcephsqlite-17.2.7-1008.ge47ff4b8.el9. 448 kB/s | 163 kB 00:00 2024-06-14T22:29:48.014 INFO:teuthology.orchestra.run.smithi046.stdout:(4/4): ceph-test-17.2.7-1008.ge47ff4b8.el9.x86_ 12 MB/s | 48 MB 00:04 2024-06-14T22:29:48.016 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:29:48.017 INFO:teuthology.orchestra.run.smithi046.stdout:Total 12 MB/s | 48 MB 00:04 2024-06-14T22:29:48.045 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-06-14T22:29:48.066 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-06-14T22:29:48.066 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-06-14T22:29:48.450 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-06-14T22:29:48.450 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-06-14T22:29:48.664 INFO:teuthology.orchestra.run.smithi178.stdout:(4/4): ceph-test-17.2.7-1008.ge47ff4b8.el9.x86_ 13 MB/s | 48 MB 00:03 2024-06-14T22:29:48.666 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:29:48.667 INFO:teuthology.orchestra.run.smithi178.stdout:Total 13 MB/s | 48 MB 00:03 2024-06-14T22:29:48.693 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-06-14T22:29:48.713 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-06-14T22:29:48.713 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-06-14T22:29:48.987 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-06-14T22:29:49.079 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-06-14T22:29:49.079 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-06-14T22:29:49.080 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-06-14T22:29:49.132 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-06-14T22:29:49.158 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephsqlite-2:17.2.7-1008.ge47ff4b8.el9.x86_64 3/4 2024-06-14T22:29:49.604 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-06-14T22:29:49.707 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-06-14T22:29:49.754 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-06-14T22:29:49.783 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephsqlite-2:17.2.7-1008.ge47ff4b8.el9.x86_64 3/4 2024-06-14T22:29:53.787 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1008.ge47ff4b8.el9.x86_64 3/4 2024-06-14T22:29:53.893 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-test-2:17.2.7-1008.ge47ff4b8.el9.x86_64 4/4 2024-06-14T22:29:54.211 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1008.ge47ff4b8.el9.x86_64 3/4 2024-06-14T22:29:54.289 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-test-2:17.2.7-1008.ge47ff4b8.el9.x86_64 4/4 2024-06-14T22:29:56.005 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-test-2:17.2.7-1008.ge47ff4b8.el9.x86_64 4/4 2024-06-14T22:29:56.005 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-test-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/4 2024-06-14T22:29:56.006 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephsqlite-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2/4 2024-06-14T22:29:56.006 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-06-14T22:29:56.289 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-06-14T22:29:56.289 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:29:56.289 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-06-14T22:29:56.289 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:56.289 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:56.289 INFO:teuthology.orchestra.run.smithi046.stdout: socat-1.7.4.1-6.el9.x86_64 2024-06-14T22:29:56.289 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-14T22:29:56.289 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:29:56.289 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:29:56.557 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-test-2:17.2.7-1008.ge47ff4b8.el9.x86_64 4/4 2024-06-14T22:29:56.557 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-test-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/4 2024-06-14T22:29:56.557 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephsqlite-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2/4 2024-06-14T22:29:56.557 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-06-14T22:29:56.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph 2024-06-14T22:29:56.853 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-06-14T22:29:56.853 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:29:56.854 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-06-14T22:29:56.854 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:56.854 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:29:56.854 INFO:teuthology.orchestra.run.smithi178.stdout: socat-1.7.4.1-6.el9.x86_64 2024-06-14T22:29:56.854 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-14T22:29:56.854 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:29:56.854 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:29:57.142 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph 2024-06-14T22:29:57.154 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:42 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:29:57.290 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:29:57.293 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:29:57.293 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-06-14T22:29:57.293 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:29:57.293 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-06-14T22:29:57.293 INFO:teuthology.orchestra.run.smithi046.stdout: ceph x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 6.5 k 2024-06-14T22:29:57.293 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-06-14T22:29:57.293 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 2.1 M 2024-06-14T22:29:57.293 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 1.4 M 2024-06-14T22:29:57.293 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1008.ge47ff4b8.el9 ceph-noarch 240 k 2024-06-14T22:29:57.293 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 3.9 M 2024-06-14T22:29:57.294 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 15 M 2024-06-14T22:29:57.294 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-14T22:29:57.294 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-14T22:29:57.294 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-14T22:29:57.294 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-14T22:29:57.294 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-14T22:29:57.294 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-14T22:29:57.294 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-14T22:29:57.294 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections 2024-06-14T22:29:57.294 INFO:teuthology.orchestra.run.smithi046.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-06-14T22:29:57.294 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-14T22:29:57.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-14T22:29:57.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-14T22:29:57.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-14T22:29:57.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-14T22:29:57.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-14T22:29:57.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-14T22:29:57.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-14T22:29:57.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-14T22:29:57.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-14T22:29:57.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-14T22:29:57.295 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-14T22:29:57.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-14T22:29:57.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-14T22:29:57.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-14T22:29:57.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-14T22:29:57.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-14T22:29:57.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-14T22:29:57.296 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-14T22:29:57.296 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:29:57.296 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-06-14T22:29:57.296 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:29:57.296 INFO:teuthology.orchestra.run.smithi046.stdout:Install 33 Packages 2024-06-14T22:29:57.296 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:29:57.297 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 27 M 2024-06-14T22:29:57.297 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 96 M 2024-06-14T22:29:57.297 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-06-14T22:29:57.729 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:40 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:29:57.858 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:29:57.861 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:29:57.861 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-06-14T22:29:57.861 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:29:57.861 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-06-14T22:29:57.861 INFO:teuthology.orchestra.run.smithi178.stdout: ceph x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 6.5 k 2024-06-14T22:29:57.861 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-06-14T22:29:57.861 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 2.1 M 2024-06-14T22:29:57.861 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 1.4 M 2024-06-14T22:29:57.861 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1008.ge47ff4b8.el9 ceph-noarch 240 k 2024-06-14T22:29:57.861 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 3.9 M 2024-06-14T22:29:57.861 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 15 M 2024-06-14T22:29:57.861 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-14T22:29:57.862 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-14T22:29:57.862 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-14T22:29:57.862 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-14T22:29:57.862 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-14T22:29:57.862 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-14T22:29:57.862 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-14T22:29:57.862 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections 2024-06-14T22:29:57.862 INFO:teuthology.orchestra.run.smithi178.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-06-14T22:29:57.862 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-14T22:29:57.862 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-14T22:29:57.862 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-14T22:29:57.863 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-14T22:29:57.863 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-14T22:29:57.863 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-14T22:29:57.863 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-14T22:29:57.863 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-14T22:29:57.863 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-14T22:29:57.863 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-14T22:29:57.863 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-14T22:29:57.863 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-14T22:29:57.863 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-14T22:29:57.863 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-14T22:29:57.864 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-14T22:29:57.864 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-14T22:29:57.864 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-14T22:29:57.864 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-14T22:29:57.864 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-14T22:29:57.864 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:29:57.864 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-06-14T22:29:57.864 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:29:57.864 INFO:teuthology.orchestra.run.smithi178.stdout:Install 33 Packages 2024-06-14T22:29:57.864 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:29:57.865 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 27 M 2024-06-14T22:29:57.865 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 96 M 2024-06-14T22:29:57.865 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-06-14T22:29:58.048 INFO:teuthology.orchestra.run.smithi046.stdout:(1/33): ceph-17.2.7-1008.ge47ff4b8.el9.x86_64.r 25 kB/s | 6.5 kB 00:00 2024-06-14T22:29:58.298 INFO:teuthology.orchestra.run.smithi046.stdout:(2/33): ceph-mgr-17.2.7-1008.ge47ff4b8.el9.x86_ 2.8 MB/s | 1.4 MB 00:00 2024-06-14T22:29:58.340 INFO:teuthology.orchestra.run.smithi046.stdout:(3/33): ceph-mds-17.2.7-1008.ge47ff4b8.el9.x86_ 3.8 MB/s | 2.1 MB 00:00 2024-06-14T22:29:58.424 INFO:teuthology.orchestra.run.smithi046.stdout:(4/33): ceph-mgr-modules-core-17.2.7-1008.ge47f 2.8 MB/s | 240 kB 00:00 2024-06-14T22:29:58.457 INFO:teuthology.orchestra.run.smithi178.stdout:(1/33): ceph-17.2.7-1008.ge47ff4b8.el9.x86_64.r 36 kB/s | 6.5 kB 00:00 2024-06-14T22:29:58.558 INFO:teuthology.orchestra.run.smithi046.stdout:(5/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-06-14T22:29:58.774 INFO:teuthology.orchestra.run.smithi046.stdout:(6/33): python3-cryptography-36.0.1-4.el9.x86_6 5.7 MB/s | 1.2 MB 00:00 2024-06-14T22:29:58.833 INFO:teuthology.orchestra.run.smithi046.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-06-14T22:29:58.892 INFO:teuthology.orchestra.run.smithi046.stdout:(8/33): ceph-mon-17.2.7-1008.ge47ff4b8.el9.x86_ 4.6 MB/s | 3.9 MB 00:00 2024-06-14T22:29:58.917 INFO:teuthology.orchestra.run.smithi046.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 1.6 MB/s | 135 kB 00:00 2024-06-14T22:29:59.017 INFO:teuthology.orchestra.run.smithi046.stdout:(10/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.1 MB/s | 215 kB 00:00 2024-06-14T22:29:59.043 INFO:teuthology.orchestra.run.smithi046.stdout:(11/33): python3-requests-2.25.1-8.el9.noarch.r 829 kB/s | 125 kB 00:00 2024-06-14T22:29:59.085 INFO:teuthology.orchestra.run.smithi046.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 838 kB/s | 35 kB 00:00 2024-06-14T22:29:59.110 INFO:teuthology.orchestra.run.smithi046.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-06-14T22:29:59.135 INFO:teuthology.orchestra.run.smithi046.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-14T22:29:59.160 INFO:teuthology.orchestra.run.smithi046.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-06-14T22:29:59.232 INFO:teuthology.orchestra.run.smithi178.stdout:(2/33): ceph-mds-17.2.7-1008.ge47ff4b8.el9.x86_ 2.2 MB/s | 2.1 MB 00:00 2024-06-14T22:29:59.369 INFO:teuthology.orchestra.run.smithi046.stdout:(16/33): ceph-osd-17.2.7-1008.ge47ff4b8.el9.x86 14 MB/s | 15 MB 00:01 2024-06-14T22:29:59.394 INFO:teuthology.orchestra.run.smithi046.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 168 kB/s | 43 kB 00:00 2024-06-14T22:29:59.444 INFO:teuthology.orchestra.run.smithi046.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 434 kB/s | 11 kB 00:00 2024-06-14T22:29:59.449 INFO:teuthology.orchestra.run.smithi178.stdout:(3/33): ceph-mgr-17.2.7-1008.ge47ff4b8.el9.x86_ 1.2 MB/s | 1.4 MB 00:01 2024-06-14T22:29:59.470 INFO:teuthology.orchestra.run.smithi046.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 556 kB/s | 172 kB 00:00 2024-06-14T22:29:59.495 INFO:teuthology.orchestra.run.smithi046.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 371 kB/s | 18 kB 00:00 2024-06-14T22:29:59.520 INFO:teuthology.orchestra.run.smithi046.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 463 kB/s | 23 kB 00:00 2024-06-14T22:29:59.541 INFO:teuthology.orchestra.run.smithi178.stdout:(4/33): ceph-mon-17.2.7-1008.ge47ff4b8.el9.x86_ 3.6 MB/s | 3.9 MB 00:01 2024-06-14T22:29:59.545 INFO:teuthology.orchestra.run.smithi046.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-06-14T22:29:59.571 INFO:teuthology.orchestra.run.smithi046.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-06-14T22:29:59.591 INFO:teuthology.orchestra.run.smithi178.stdout:(5/33): ceph-mgr-modules-core-17.2.7-1008.ge47f 1.7 MB/s | 240 kB 00:00 2024-06-14T22:29:59.596 INFO:teuthology.orchestra.run.smithi046.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 919 kB/s | 46 kB 00:00 2024-06-14T22:29:59.621 INFO:teuthology.orchestra.run.smithi046.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-06-14T22:29:59.647 INFO:teuthology.orchestra.run.smithi046.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 656 kB/s | 16 kB 00:00 2024-06-14T22:29:59.672 INFO:teuthology.orchestra.run.smithi046.stdout:(27/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.2 MB/s | 358 kB 00:00 2024-06-14T22:29:59.697 INFO:teuthology.orchestra.run.smithi046.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-06-14T22:29:59.722 INFO:teuthology.orchestra.run.smithi046.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 714 kB/s | 36 kB 00:00 2024-06-14T22:29:59.747 INFO:teuthology.orchestra.run.smithi046.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-06-14T22:29:59.773 INFO:teuthology.orchestra.run.smithi046.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 802 kB/s | 20 kB 00:00 2024-06-14T22:29:59.775 INFO:teuthology.orchestra.run.smithi178.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-06-14T22:29:59.798 INFO:teuthology.orchestra.run.smithi046.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-06-14T22:29:59.823 INFO:teuthology.orchestra.run.smithi046.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-06-14T22:29:59.828 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:29:59.828 INFO:teuthology.orchestra.run.smithi046.stdout:Total 11 MB/s | 27 MB 00:02 2024-06-14T22:29:59.833 INFO:teuthology.orchestra.run.smithi178.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-06-14T22:29:59.875 INFO:teuthology.orchestra.run.smithi178.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-06-14T22:29:59.925 INFO:teuthology.orchestra.run.smithi178.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 2.4 MB/s | 125 kB 00:00 2024-06-14T22:29:59.946 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-06-14T22:29:59.959 INFO:teuthology.orchestra.run.smithi178.stdout:(10/33): python3-cryptography-36.0.1-4.el9.x86_ 3.3 MB/s | 1.2 MB 00:00 2024-06-14T22:29:59.985 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-06-14T22:29:59.985 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-06-14T22:30:00.009 INFO:teuthology.orchestra.run.smithi178.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-06-14T22:30:00.357 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-06-14T22:30:00.357 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-06-14T22:30:00.412 INFO:teuthology.orchestra.run.smithi178.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 86 kB/s | 35 kB 00:00 2024-06-14T22:30:00.479 INFO:teuthology.orchestra.run.smithi178.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 331 kB/s | 172 kB 00:00 2024-06-14T22:30:00.629 INFO:teuthology.orchestra.run.smithi178.stdout:(14/33): ceph-osd-17.2.7-1008.ge47ff4b8.el9.x86 11 MB/s | 15 MB 00:01 2024-06-14T22:30:00.679 INFO:teuthology.orchestra.run.smithi178.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 191 kB/s | 51 kB 00:00 2024-06-14T22:30:00.771 INFO:teuthology.orchestra.run.smithi178.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 143 kB/s | 42 kB 00:00 2024-06-14T22:30:00.797 INFO:teuthology.orchestra.run.smithi178.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 304 kB/s | 43 kB 00:00 2024-06-14T22:30:00.847 INFO:teuthology.orchestra.run.smithi178.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-06-14T22:30:00.872 INFO:teuthology.orchestra.run.smithi178.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 209 kB/s | 11 kB 00:00 2024-06-14T22:30:00.897 INFO:teuthology.orchestra.run.smithi178.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 783 kB/s | 18 kB 00:00 2024-06-14T22:30:00.923 INFO:teuthology.orchestra.run.smithi178.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 484 kB/s | 23 kB 00:00 2024-06-14T22:30:00.948 INFO:teuthology.orchestra.run.smithi178.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-06-14T22:30:00.973 INFO:teuthology.orchestra.run.smithi178.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-06-14T22:30:00.998 INFO:teuthology.orchestra.run.smithi178.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 919 kB/s | 46 kB 00:00 2024-06-14T22:30:01.024 INFO:teuthology.orchestra.run.smithi178.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-06-14T22:30:01.049 INFO:teuthology.orchestra.run.smithi178.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 655 kB/s | 16 kB 00:00 2024-06-14T22:30:01.074 INFO:teuthology.orchestra.run.smithi178.stdout:(27/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.2 MB/s | 358 kB 00:00 2024-06-14T22:30:01.100 INFO:teuthology.orchestra.run.smithi178.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-06-14T22:30:01.125 INFO:teuthology.orchestra.run.smithi178.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 710 kB/s | 36 kB 00:00 2024-06-14T22:30:01.150 INFO:teuthology.orchestra.run.smithi178.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-06-14T22:30:01.177 INFO:teuthology.orchestra.run.smithi178.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 799 kB/s | 20 kB 00:00 2024-06-14T22:30:01.201 INFO:teuthology.orchestra.run.smithi178.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-06-14T22:30:01.226 INFO:teuthology.orchestra.run.smithi178.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-06-14T22:30:01.231 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:01.231 INFO:teuthology.orchestra.run.smithi178.stdout:Total 8.1 MB/s | 27 MB 00:03 2024-06-14T22:30:01.345 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-06-14T22:30:01.383 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-06-14T22:30:01.383 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-06-14T22:30:01.526 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-06-14T22:30:01.576 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-06-14T22:30:01.609 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-06-14T22:30:01.641 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-06-14T22:30:01.739 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-06-14T22:30:01.744 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-06-14T22:30:01.744 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-06-14T22:30:01.824 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-06-14T22:30:01.871 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-06-14T22:30:01.915 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-06-14T22:30:01.953 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-06-14T22:30:01.987 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-06-14T22:30:02.020 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-06-14T22:30:02.061 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-06-14T22:30:02.105 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-06-14T22:30:02.147 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-06-14T22:30:02.186 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-06-14T22:30:02.261 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-06-14T22:30:02.420 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-06-14T22:30:02.516 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-06-14T22:30:02.573 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-06-14T22:30:02.635 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-06-14T22:30:02.693 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-06-14T22:30:02.766 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-06-14T22:30:02.879 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-06-14T22:30:02.946 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-06-14T22:30:02.975 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-06-14T22:30:02.978 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-06-14T22:30:03.013 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-06-14T22:30:03.022 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-06-14T22:30:03.103 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-06-14T22:30:03.104 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-06-14T22:30:03.183 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-06-14T22:30:03.193 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-06-14T22:30:03.231 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-06-14T22:30:03.234 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-06-14T22:30:03.266 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-06-14T22:30:03.295 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-06-14T22:30:03.330 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-06-14T22:30:03.363 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-06-14T22:30:03.369 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-06-14T22:30:03.404 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-06-14T22:30:03.447 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-06-14T22:30:03.481 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-06-14T22:30:03.520 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-06-14T22:30:03.532 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1008.ge47ff4b8.el9. 28/33 2024-06-14T22:30:03.556 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-2:17.2.7-1008.ge47ff4b8.el9.x86_64 29/33 2024-06-14T22:30:03.584 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1008.ge47ff4b8.el9.x86_64 29/33 2024-06-14T22:30:03.585 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-14T22:30:03.585 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-14T22:30:03.585 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-14T22:30:03.585 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-14T22:30:03.585 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:03.594 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-06-14T22:30:03.759 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-06-14T22:30:03.845 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-06-14T22:30:03.906 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-06-14T22:30:03.945 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-06-14T22:30:04.003 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-06-14T22:30:04.058 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-06-14T22:30:04.252 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-06-14T22:30:04.308 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-06-14T22:30:04.386 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-06-14T22:30:04.475 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-06-14T22:30:04.517 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-06-14T22:30:04.641 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-06-14T22:30:04.803 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1008.ge47ff4b8.el9. 28/33 2024-06-14T22:30:04.826 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-2:17.2.7-1008.ge47ff4b8.el9.x86_64 29/33 2024-06-14T22:30:04.862 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1008.ge47ff4b8.el9.x86_64 29/33 2024-06-14T22:30:04.862 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-14T22:30:04.862 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-14T22:30:04.862 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-14T22:30:04.862 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-14T22:30:04.862 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:04.974 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-osd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 30/33 2024-06-14T22:30:05.003 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-osd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 30/33 2024-06-14T22:30:05.003 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-14T22:30:05.004 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-14T22:30:05.004 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-14T22:30:05.004 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-14T22:30:05.004 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:05.391 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mon-2:17.2.7-1008.ge47ff4b8.el9.x86_64 31/33 2024-06-14T22:30:05.416 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mon-2:17.2.7-1008.ge47ff4b8.el9.x86_64 31/33 2024-06-14T22:30:05.416 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-14T22:30:05.416 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-14T22:30:05.417 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-14T22:30:05.417 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-14T22:30:05.417 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:05.625 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mds-2:17.2.7-1008.ge47ff4b8.el9.x86_64 32/33 2024-06-14T22:30:05.650 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mds-2:17.2.7-1008.ge47ff4b8.el9.x86_64 32/33 2024-06-14T22:30:05.650 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-14T22:30:05.650 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-14T22:30:05.650 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-14T22:30:05.650 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-14T22:30:05.650 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:05.715 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-2:17.2.7-1008.ge47ff4b8.el9.x86_64 33/33 2024-06-14T22:30:06.185 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-osd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 30/33 2024-06-14T22:30:06.213 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-osd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 30/33 2024-06-14T22:30:06.213 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-14T22:30:06.213 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-14T22:30:06.213 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-14T22:30:06.213 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-14T22:30:06.213 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:06.586 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mon-2:17.2.7-1008.ge47ff4b8.el9.x86_64 31/33 2024-06-14T22:30:06.610 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mon-2:17.2.7-1008.ge47ff4b8.el9.x86_64 31/33 2024-06-14T22:30:06.610 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-14T22:30:06.610 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-14T22:30:06.610 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-14T22:30:06.610 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-14T22:30:06.610 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:06.820 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mds-2:17.2.7-1008.ge47ff4b8.el9.x86_64 32/33 2024-06-14T22:30:06.843 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mds-2:17.2.7-1008.ge47ff4b8.el9.x86_64 32/33 2024-06-14T22:30:06.843 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-14T22:30:06.843 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-14T22:30:06.843 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-14T22:30:06.843 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-14T22:30:06.843 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:06.916 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-2:17.2.7-1008.ge47ff4b8.el9.x86_64 33/33 2024-06-14T22:30:07.472 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-2:17.2.7-1008.ge47ff4b8.el9.x86_64 33/33 2024-06-14T22:30:07.472 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/33 2024-06-14T22:30:07.472 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mds-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2/33 2024-06-14T22:30:07.472 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-2:17.2.7-1008.ge47ff4b8.el9.x86_64 3/33 2024-06-14T22:30:07.472 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mon-2:17.2.7-1008.ge47ff4b8.el9.x86_64 4/33 2024-06-14T22:30:07.472 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-osd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 5/33 2024-06-14T22:30:07.473 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1008.ge47ff4b8.el9. 6/33 2024-06-14T22:30:07.473 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-06-14T22:30:07.473 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-06-14T22:30:07.473 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-06-14T22:30:07.473 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-06-14T22:30:07.473 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-06-14T22:30:07.473 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-06-14T22:30:07.473 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-06-14T22:30:07.473 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-06-14T22:30:07.473 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-06-14T22:30:07.473 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-06-14T22:30:07.473 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-06-14T22:30:07.473 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-06-14T22:30:07.474 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-06-14T22:30:07.474 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-06-14T22:30:07.474 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-06-14T22:30:07.474 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-06-14T22:30:07.474 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-06-14T22:30:07.474 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-06-14T22:30:07.474 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-06-14T22:30:07.475 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-06-14T22:30:07.475 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-06-14T22:30:07.475 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-06-14T22:30:07.475 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-06-14T22:30:07.475 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-06-14T22:30:07.475 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-06-14T22:30:07.475 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-06-14T22:30:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-06-14T22:30:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-06-14T22:30:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:30:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:30:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:30:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core-2:17.2.7-1008.ge47ff4b8.el9.noarch 2024-06-14T22:30:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:30:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:30:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-14T22:30:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-14T22:30:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-14T22:30:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-14T22:30:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-14T22:30:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-14T22:30:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-14T22:30:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-14T22:30:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-14T22:30:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-14T22:30:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-14T22:30:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-14T22:30:08.471 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-14T22:30:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-14T22:30:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-14T22:30:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply-3.11-14.el9.noarch 2024-06-14T22:30:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-14T22:30:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-14T22:30:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-14T22:30:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-14T22:30:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-14T22:30:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-14T22:30:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-14T22:30:08.472 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-14T22:30:08.473 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-14T22:30:08.473 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-14T22:30:08.473 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-14T22:30:08.473 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:08.473 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:08.552 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-2:17.2.7-1008.ge47ff4b8.el9.x86_64 33/33 2024-06-14T22:30:08.553 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/33 2024-06-14T22:30:08.553 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mds-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2/33 2024-06-14T22:30:08.553 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-2:17.2.7-1008.ge47ff4b8.el9.x86_64 3/33 2024-06-14T22:30:08.553 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mon-2:17.2.7-1008.ge47ff4b8.el9.x86_64 4/33 2024-06-14T22:30:08.553 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-osd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 5/33 2024-06-14T22:30:08.553 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1008.ge47ff4b8.el9. 6/33 2024-06-14T22:30:08.553 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-06-14T22:30:08.553 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-06-14T22:30:08.553 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-06-14T22:30:08.553 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-06-14T22:30:08.553 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-06-14T22:30:08.553 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-06-14T22:30:08.553 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-06-14T22:30:08.554 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-06-14T22:30:08.554 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-06-14T22:30:08.554 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-06-14T22:30:08.554 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-06-14T22:30:08.554 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-06-14T22:30:08.554 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-06-14T22:30:08.554 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-06-14T22:30:08.554 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-06-14T22:30:08.554 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-06-14T22:30:08.554 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-06-14T22:30:08.554 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-06-14T22:30:08.555 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-06-14T22:30:08.555 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-06-14T22:30:08.555 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-06-14T22:30:08.555 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-06-14T22:30:08.556 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-06-14T22:30:08.556 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-06-14T22:30:08.556 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-06-14T22:30:08.556 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-06-14T22:30:08.803 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-base 2024-06-14T22:30:09.405 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:54 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:09.430 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-06-14T22:30:09.430 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:09.430 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-06-14T22:30:09.430 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:30:09.430 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:30:09.430 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:30:09.430 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core-2:17.2.7-1008.ge47ff4b8.el9.noarch 2024-06-14T22:30:09.430 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:30:09.430 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:30:09.430 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-14T22:30:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-14T22:30:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-14T22:30:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-14T22:30:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-14T22:30:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-14T22:30:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-14T22:30:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-14T22:30:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-14T22:30:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-14T22:30:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-14T22:30:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-14T22:30:09.431 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-14T22:30:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-14T22:30:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-14T22:30:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply-3.11-14.el9.noarch 2024-06-14T22:30:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-14T22:30:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-14T22:30:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-14T22:30:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-14T22:30:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-14T22:30:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-14T22:30:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-14T22:30:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-14T22:30:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-14T22:30:09.432 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-14T22:30:09.433 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-14T22:30:09.433 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:09.433 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:30:09.492 INFO:teuthology.orchestra.run.smithi046.stdout:Package ceph-base-2:17.2.7-1008.ge47ff4b8.el9.x86_64 is already installed. 2024-06-14T22:30:09.537 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:09.538 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-06-14T22:30:09.538 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:09.607 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install cephadm 2024-06-14T22:30:09.803 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-base 2024-06-14T22:30:10.204 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:55 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:10.334 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:10.335 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:10.335 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-06-14T22:30:10.335 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:10.335 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-06-14T22:30:10.336 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm noarch 2:17.2.7-1008.ge47ff4b8.el9 ceph-noarch 82 k 2024-06-14T22:30:10.336 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:10.336 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-06-14T22:30:10.336 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:10.336 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-06-14T22:30:10.336 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:10.336 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 82 k 2024-06-14T22:30:10.336 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 372 k 2024-06-14T22:30:10.336 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-06-14T22:30:10.384 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:53 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:30:10.469 INFO:teuthology.orchestra.run.smithi178.stdout:Package ceph-base-2:17.2.7-1008.ge47ff4b8.el9.x86_64 is already installed. 2024-06-14T22:30:10.511 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:30:10.512 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-06-14T22:30:10.512 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:30:10.532 INFO:teuthology.orchestra.run.smithi046.stdout:cephadm-17.2.7-1008.ge47ff4b8.el9.noarch.rpm 420 kB/s | 82 kB 00:00 2024-06-14T22:30:10.532 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:10.532 INFO:teuthology.orchestra.run.smithi046.stdout:Total 417 kB/s | 82 kB 00:00 2024-06-14T22:30:10.533 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-06-14T22:30:10.538 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-06-14T22:30:10.538 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-06-14T22:30:10.540 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-06-14T22:30:10.540 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-06-14T22:30:10.584 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install cephadm 2024-06-14T22:30:10.612 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-06-14T22:30:11.066 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephadm-2:17.2.7-1008.ge47ff4b8.el9.noarch 1/1 2024-06-14T22:30:11.157 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:54 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:30:11.171 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephadm-2:17.2.7-1008.ge47ff4b8.el9.noarch 1/1 2024-06-14T22:30:11.283 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:30:11.284 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:11.284 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-06-14T22:30:11.284 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:11.284 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-06-14T22:30:11.284 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm noarch 2:17.2.7-1008.ge47ff4b8.el9 ceph-noarch 82 k 2024-06-14T22:30:11.284 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:11.284 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-06-14T22:30:11.284 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:11.284 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-06-14T22:30:11.284 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:11.285 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 82 k 2024-06-14T22:30:11.285 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 372 k 2024-06-14T22:30:11.285 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-06-14T22:30:11.488 INFO:teuthology.orchestra.run.smithi178.stdout:cephadm-17.2.7-1008.ge47ff4b8.el9.noarch.rpm 404 kB/s | 82 kB 00:00 2024-06-14T22:30:11.488 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:11.489 INFO:teuthology.orchestra.run.smithi178.stdout:Total 401 kB/s | 82 kB 00:00 2024-06-14T22:30:11.489 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-06-14T22:30:11.494 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-06-14T22:30:11.494 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-06-14T22:30:11.496 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-06-14T22:30:11.496 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-06-14T22:30:11.563 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-06-14T22:30:11.664 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephadm-2:17.2.7-1008.ge47ff4b8.el9.noarch 1/1 2024-06-14T22:30:11.920 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephadm-2:17.2.7-1008.ge47ff4b8.el9.noarch 1/1 2024-06-14T22:30:11.920 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:11.920 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-06-14T22:30:11.921 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm-2:17.2.7-1008.ge47ff4b8.el9.noarch 2024-06-14T22:30:11.921 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:11.921 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:12.021 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:17.2.7-1008.ge47ff4b8.el9.noarch 1/1 2024-06-14T22:30:12.109 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephadm-2:17.2.7-1008.ge47ff4b8.el9.noarch 1/1 2024-06-14T22:30:12.120 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-immutable-object-cache 2024-06-14T22:30:12.450 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:17.2.7-1008.ge47ff4b8.el9.noarch 1/1 2024-06-14T22:30:12.706 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephadm-2:17.2.7-1008.ge47ff4b8.el9.noarch 1/1 2024-06-14T22:30:12.706 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:12.706 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-06-14T22:30:12.706 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm-2:17.2.7-1008.ge47ff4b8.el9.noarch 2024-06-14T22:30:12.706 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:12.706 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:30:12.719 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:00:57 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:12.852 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:12.852 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:12.853 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repo Size 2024-06-14T22:30:12.853 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:12.853 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-06-14T22:30:12.853 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 137 k 2024-06-14T22:30:12.853 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:12.854 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-06-14T22:30:12.854 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:12.854 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-06-14T22:30:12.854 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:12.854 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 137 k 2024-06-14T22:30:12.854 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 379 k 2024-06-14T22:30:12.854 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-06-14T22:30:12.934 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-immutable-object-cache 2024-06-14T22:30:13.087 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-immutable-object-cache-17.2.7-1008.ge47ff4 588 kB/s | 137 kB 00:00 2024-06-14T22:30:13.088 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:13.088 INFO:teuthology.orchestra.run.smithi046.stdout:Total 584 kB/s | 137 kB 00:00 2024-06-14T22:30:13.088 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-06-14T22:30:13.096 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-06-14T22:30:13.096 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-06-14T22:30:13.157 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-06-14T22:30:13.158 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-06-14T22:30:13.311 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-06-14T22:30:13.380 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1008.ge47ff4b8. 1/1 2024-06-14T22:30:13.406 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1008.ge47ff4b8. 1/1 2024-06-14T22:30:13.406 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-14T22:30:13.406 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-14T22:30:13.407 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:13.521 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:56 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:30:13.650 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:30:13.650 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:13.650 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repo Size 2024-06-14T22:30:13.650 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:13.651 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-06-14T22:30:13.651 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 137 k 2024-06-14T22:30:13.651 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:13.651 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-06-14T22:30:13.651 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:13.651 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-06-14T22:30:13.651 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:13.652 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 137 k 2024-06-14T22:30:13.652 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 379 k 2024-06-14T22:30:13.652 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-06-14T22:30:14.012 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-immutable-object-cache-17.2.7-1008.ge47ff4 381 kB/s | 137 kB 00:00 2024-06-14T22:30:14.013 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:14.013 INFO:teuthology.orchestra.run.smithi178.stdout:Total 379 kB/s | 137 kB 00:00 2024-06-14T22:30:14.013 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-06-14T22:30:14.021 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-06-14T22:30:14.021 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-06-14T22:30:14.077 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-06-14T22:30:14.078 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-06-14T22:30:14.235 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-06-14T22:30:14.305 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1008.ge47ff4b8. 1/1 2024-06-14T22:30:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1008.ge47ff4b8. 1/1 2024-06-14T22:30:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:14.313 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-06-14T22:30:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:30:14.313 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:14.314 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:14.330 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1008.ge47ff4b8. 1/1 2024-06-14T22:30:14.330 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-14T22:30:14.330 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-14T22:30:14.330 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:14.507 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr 2024-06-14T22:30:15.111 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:00 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:15.197 INFO:teuthology.orchestra.run.smithi046.stdout:Package ceph-mgr-2:17.2.7-1008.ge47ff4b8.el9.x86_64 is already installed. 2024-06-14T22:30:15.241 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:15.243 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-06-14T22:30:15.243 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:15.256 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1008.ge47ff4b8. 1/1 2024-06-14T22:30:15.256 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:15.257 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-06-14T22:30:15.257 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:30:15.257 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:15.257 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:30:15.307 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr-dashboard 2024-06-14T22:30:15.484 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr 2024-06-14T22:30:15.901 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:00 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:16.038 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:16.039 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:16.039 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-06-14T22:30:16.039 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:16.039 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-06-14T22:30:16.039 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1008.ge47ff4b8.el9 ceph-noarch 3.4 M 2024-06-14T22:30:16.039 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-06-14T22:30:16.039 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1008.ge47ff4b8.el9 ceph-noarch 24 k 2024-06-14T22:30:16.040 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1008.ge47ff4b8.el9 ceph-noarch 15 k 2024-06-14T22:30:16.040 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-06-14T22:30:16.040 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-14T22:30:16.040 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-14T22:30:16.040 INFO:teuthology.orchestra.run.smithi046.stdout:Installing weak dependencies: 2024-06-14T22:30:16.040 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-06-14T22:30:16.040 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:16.040 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-06-14T22:30:16.040 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:16.040 INFO:teuthology.orchestra.run.smithi046.stdout:Install 7 Packages 2024-06-14T22:30:16.040 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:16.041 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 3.7 M 2024-06-14T22:30:16.041 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 80 M 2024-06-14T22:30:16.041 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-06-14T22:30:16.071 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:59 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:30:16.158 INFO:teuthology.orchestra.run.smithi178.stdout:Package ceph-mgr-2:17.2.7-1008.ge47ff4b8.el9.x86_64 is already installed. 2024-06-14T22:30:16.200 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:30:16.201 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-06-14T22:30:16.201 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:30:16.285 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-dashboard 2024-06-14T22:30:16.666 INFO:teuthology.orchestra.run.smithi046.stdout:(1/7): ceph-prometheus-alerts-17.2.7-1008.ge47f 75 kB/s | 15 kB 00:00 2024-06-14T22:30:16.691 INFO:teuthology.orchestra.run.smithi046.stdout:(2/7): ceph-grafana-dashboards-17.2.7-1008.ge47 110 kB/s | 24 kB 00:00 2024-06-14T22:30:16.775 INFO:teuthology.orchestra.run.smithi046.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 84 kB/s | 9.0 kB 00:00 2024-06-14T22:30:16.800 INFO:teuthology.orchestra.run.smithi046.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-06-14T22:30:16.825 INFO:teuthology.orchestra.run.smithi046.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 305 kB/s | 41 kB 00:00 2024-06-14T22:30:16.842 INFO:teuthology.orchestra.run.smithi046.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 4.4 MB/s | 188 kB 00:00 2024-06-14T22:30:16.858 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:59 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:30:16.986 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:30:16.987 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:16.987 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-06-14T22:30:16.987 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:16.988 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-06-14T22:30:16.988 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1008.ge47ff4b8.el9 ceph-noarch 3.4 M 2024-06-14T22:30:16.988 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-06-14T22:30:16.988 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1008.ge47ff4b8.el9 ceph-noarch 24 k 2024-06-14T22:30:16.988 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1008.ge47ff4b8.el9 ceph-noarch 15 k 2024-06-14T22:30:16.988 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-06-14T22:30:16.988 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-14T22:30:16.988 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-14T22:30:16.988 INFO:teuthology.orchestra.run.smithi178.stdout:Installing weak dependencies: 2024-06-14T22:30:16.988 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-06-14T22:30:16.988 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:16.988 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-06-14T22:30:16.988 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:16.989 INFO:teuthology.orchestra.run.smithi178.stdout:Install 7 Packages 2024-06-14T22:30:16.989 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:16.989 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 3.7 M 2024-06-14T22:30:16.989 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 80 M 2024-06-14T22:30:16.989 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-06-14T22:30:17.234 INFO:teuthology.orchestra.run.smithi178.stdout:(1/7): ceph-prometheus-alerts-17.2.7-1008.ge47f 73 kB/s | 15 kB 00:00 2024-06-14T22:30:17.284 INFO:teuthology.orchestra.run.smithi178.stdout:(2/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 181 kB/s | 9.0 kB 00:00 2024-06-14T22:30:17.318 INFO:teuthology.orchestra.run.smithi178.stdout:(3/7): python3-jwt-2.4.0-1.el9.noarch.rpm 1.2 MB/s | 41 kB 00:00 2024-06-14T22:30:17.335 INFO:teuthology.orchestra.run.smithi178.stdout:(4/7): ceph-grafana-dashboards-17.2.7-1008.ge47 81 kB/s | 24 kB 00:00 2024-06-14T22:30:17.352 INFO:teuthology.orchestra.run.smithi178.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 911 kB/s | 31 kB 00:00 2024-06-14T22:30:17.410 INFO:teuthology.orchestra.run.smithi178.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 2.5 MB/s | 188 kB 00:00 2024-06-14T22:30:17.651 INFO:teuthology.orchestra.run.smithi046.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1008.ge47ff4b8 2.8 MB/s | 3.4 MB 00:01 2024-06-14T22:30:17.655 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:17.655 INFO:teuthology.orchestra.run.smithi046.stdout:Total 2.3 MB/s | 3.7 MB 00:01 2024-06-14T22:30:17.655 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-06-14T22:30:17.664 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-06-14T22:30:17.664 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-06-14T22:30:17.731 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-06-14T22:30:17.732 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-06-14T22:30:17.760 INFO:teuthology.orchestra.run.smithi178.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1008.ge47ff4b8 4.6 MB/s | 3.4 MB 00:00 2024-06-14T22:30:17.764 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:17.764 INFO:teuthology.orchestra.run.smithi178.stdout:Total 4.7 MB/s | 3.7 MB 00:00 2024-06-14T22:30:17.764 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-06-14T22:30:17.772 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-06-14T22:30:17.772 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-06-14T22:30:17.834 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-06-14T22:30:17.835 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-06-14T22:30:18.102 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-06-14T22:30:18.169 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-06-14T22:30:18.204 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-06-14T22:30:18.210 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-06-14T22:30:18.252 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-06-14T22:30:18.281 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-06-14T22:30:18.287 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-06-14T22:30:18.311 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1008.ge47ff4b8.el9.n 5/7 2024-06-14T22:30:18.321 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-06-14T22:30:18.376 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-06-14T22:30:18.408 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-06-14T22:30:18.437 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1008.ge47ff4b8.el9.n 5/7 2024-06-14T22:30:19.365 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1008.ge47ff4b8.el9. 6/7 2024-06-14T22:30:19.397 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1008.ge47ff4b8.el9.noarc 7/7 2024-06-14T22:30:19.428 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1008.ge47ff4b8.el9. 6/7 2024-06-14T22:30:19.466 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1008.ge47ff4b8.el9.noarc 7/7 2024-06-14T22:30:20.308 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1008.ge47ff4b8.el9.noarc 7/7 2024-06-14T22:30:20.309 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1008.ge47ff4b8.el9. 1/7 2024-06-14T22:30:20.309 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1008.ge47ff4b8.el9.noarc 2/7 2024-06-14T22:30:20.309 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1008.ge47ff4b8.el9.n 3/7 2024-06-14T22:30:20.309 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-06-14T22:30:20.309 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-06-14T22:30:20.309 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-06-14T22:30:20.531 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1008.ge47ff4b8.el9.noarc 7/7 2024-06-14T22:30:20.531 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1008.ge47ff4b8.el9. 1/7 2024-06-14T22:30:20.531 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1008.ge47ff4b8.el9.noarc 2/7 2024-06-14T22:30:20.531 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1008.ge47ff4b8.el9.n 3/7 2024-06-14T22:30:20.531 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-06-14T22:30:20.531 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-06-14T22:30:20.531 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-06-14T22:30:20.588 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-06-14T22:30:20.588 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:20.589 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-06-14T22:30:20.589 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards-2:17.2.7-1008.ge47ff4b8.el9.noarch 2024-06-14T22:30:20.589 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard-2:17.2.7-1008.ge47ff4b8.el9.noarch 2024-06-14T22:30:20.589 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts-2:17.2.7-1008.ge47ff4b8.el9.noarch 2024-06-14T22:30:20.589 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-06-14T22:30:20.589 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-06-14T22:30:20.589 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-14T22:30:20.589 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-14T22:30:20.589 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:20.589 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:20.788 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr-diskprediction-local 2024-06-14T22:30:20.810 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-06-14T22:30:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:20.811 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-06-14T22:30:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards-2:17.2.7-1008.ge47ff4b8.el9.noarch 2024-06-14T22:30:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard-2:17.2.7-1008.ge47ff4b8.el9.noarch 2024-06-14T22:30:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts-2:17.2.7-1008.ge47ff4b8.el9.noarch 2024-06-14T22:30:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-06-14T22:30:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-06-14T22:30:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-14T22:30:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-14T22:30:20.811 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:20.811 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:30:21.016 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-diskprediction-local 2024-06-14T22:30:21.383 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:06 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:21.511 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:21.513 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-06-14T22:30:21.513 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-06-14T22:30:21.513 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-06-14T22:30:21.513 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-06-14T22:30:21.513 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1008.ge47ff4b8.el9 ceph-noarch 7.4 M 2024-06-14T22:30:21.513 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-06-14T22:30:21.513 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-14T22:30:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-14T22:30:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-14T22:30:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-14T22:30:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-14T22:30:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-14T22:30:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-14T22:30:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-14T22:30:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-14T22:30:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-14T22:30:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-14T22:30:21.514 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:21.514 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-06-14T22:30:21.515 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-06-14T22:30:21.515 INFO:teuthology.orchestra.run.smithi046.stdout:Install 12 Packages 2024-06-14T22:30:21.515 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:21.515 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 38 M 2024-06-14T22:30:21.515 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 209 M 2024-06-14T22:30:21.515 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-06-14T22:30:21.581 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:04 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:30:21.705 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:30:21.706 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-06-14T22:30:21.706 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-06-14T22:30:21.706 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-06-14T22:30:21.706 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-06-14T22:30:21.706 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1008.ge47ff4b8.el9 ceph-noarch 7.4 M 2024-06-14T22:30:21.707 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-06-14T22:30:21.707 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-14T22:30:21.707 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-14T22:30:21.707 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-14T22:30:21.707 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-14T22:30:21.707 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-14T22:30:21.707 INFO:teuthology.orchestra.run.smithi178.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-14T22:30:21.707 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-14T22:30:21.707 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-14T22:30:21.707 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-14T22:30:21.707 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-14T22:30:21.707 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-14T22:30:21.708 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:21.708 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-06-14T22:30:21.708 INFO:teuthology.orchestra.run.smithi178.stdout:======================================================================================= 2024-06-14T22:30:21.708 INFO:teuthology.orchestra.run.smithi178.stdout:Install 12 Packages 2024-06-14T22:30:21.708 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:21.708 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 38 M 2024-06-14T22:30:21.708 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 209 M 2024-06-14T22:30:21.709 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-06-14T22:30:22.001 INFO:teuthology.orchestra.run.smithi046.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 683 kB/s | 195 kB 00:00 2024-06-14T22:30:22.076 INFO:teuthology.orchestra.run.smithi046.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 437 kB/s | 33 kB 00:00 2024-06-14T22:30:22.101 INFO:teuthology.orchestra.run.smithi046.stdout:(3/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 803 kB 00:00 2024-06-14T22:30:22.160 INFO:teuthology.orchestra.run.smithi046.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 311 kB/s | 18 kB 00:00 2024-06-14T22:30:22.202 INFO:teuthology.orchestra.run.smithi046.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 846 kB/s | 35 kB 00:00 2024-06-14T22:30:22.253 INFO:teuthology.orchestra.run.smithi178.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-06-14T22:30:22.260 INFO:teuthology.orchestra.run.smithi046.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 16 MB/s | 3.0 MB 00:00 2024-06-14T22:30:22.295 INFO:teuthology.orchestra.run.smithi178.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.7 MB/s | 803 kB 00:00 2024-06-14T22:30:22.344 INFO:teuthology.orchestra.run.smithi046.stdout:(7/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 33 MB/s | 4.7 MB 00:00 2024-06-14T22:30:22.394 INFO:teuthology.orchestra.run.smithi046.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2024-06-14T22:30:22.470 INFO:teuthology.orchestra.run.smithi046.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 5.1 MB 00:00 2024-06-14T22:30:22.495 INFO:teuthology.orchestra.run.smithi046.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.4 MB/s | 250 kB 00:00 2024-06-14T22:30:22.779 INFO:teuthology.orchestra.run.smithi046.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 53 MB/s | 16 MB 00:00 2024-06-14T22:30:22.870 INFO:teuthology.orchestra.run.smithi178.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 53 kB/s | 33 kB 00:00 2024-06-14T22:30:23.304 INFO:teuthology.orchestra.run.smithi046.stdout:(12/12): ceph-mgr-diskprediction-local-17.2.7-1 4.7 MB/s | 7.4 MB 00:01 2024-06-14T22:30:23.305 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:23.305 INFO:teuthology.orchestra.run.smithi046.stdout:Total 21 MB/s | 38 MB 00:01 2024-06-14T22:30:23.412 INFO:teuthology.orchestra.run.smithi178.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 34 kB/s | 18 kB 00:00 2024-06-14T22:30:23.471 INFO:teuthology.orchestra.run.smithi178.stdout:(5/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.5 MB/s | 3.0 MB 00:01 2024-06-14T22:30:23.563 INFO:teuthology.orchestra.run.smithi178.stdout:(6/12): ceph-mgr-diskprediction-local-17.2.7-10 5.2 MB/s | 7.4 MB 00:01 2024-06-14T22:30:23.669 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-06-14T22:30:23.697 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-06-14T22:30:23.697 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-06-14T22:30:24.130 INFO:teuthology.orchestra.run.smithi178.stdout:(7/12): openblas-0.3.21-2.el9.x86_64.rpm 49 kB/s | 35 kB 00:00 2024-06-14T22:30:24.227 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-06-14T22:30:24.227 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-06-14T22:30:24.422 INFO:teuthology.orchestra.run.smithi178.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 284 kB/s | 244 kB 00:00 2024-06-14T22:30:24.772 INFO:teuthology.orchestra.run.smithi178.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 3.6 MB/s | 4.7 MB 00:01 2024-06-14T22:30:25.012 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-06-14T22:30:25.089 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-06-14T22:30:25.127 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-06-14T22:30:25.231 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-06-14T22:30:25.270 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-06-14T22:30:25.289 INFO:teuthology.orchestra.run.smithi178.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 289 kB/s | 250 kB 00:00 2024-06-14T22:30:25.356 INFO:teuthology.orchestra.run.smithi178.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.2 MB/s | 5.1 MB 00:01 2024-06-14T22:30:25.464 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-06-14T22:30:25.494 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-06-14T22:30:25.576 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-06-14T22:30:26.264 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-06-14T22:30:26.306 INFO:teuthology.orchestra.run.smithi178.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-06-14T22:30:26.308 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:26.308 INFO:teuthology.orchestra.run.smithi178.stdout:Total 8.3 MB/s | 38 MB 00:04 2024-06-14T22:30:26.394 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-06-14T22:30:26.661 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-06-14T22:30:26.687 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-06-14T22:30:26.687 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-06-14T22:30:27.194 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-06-14T22:30:27.194 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-06-14T22:30:27.691 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-06-14T22:30:27.948 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-06-14T22:30:28.030 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-06-14T22:30:28.070 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-06-14T22:30:28.169 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-06-14T22:30:28.213 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-06-14T22:30:28.404 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-06-14T22:30:28.446 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-06-14T22:30:28.527 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-06-14T22:30:28.779 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-06-14T22:30:28.809 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1008.ge47ff 12/12 2024-06-14T22:30:29.184 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-06-14T22:30:29.301 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-06-14T22:30:30.544 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-06-14T22:30:31.333 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1008.ge47ff 12/12 2024-06-14T22:30:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1008.ge47ff 1/12 2024-06-14T22:30:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-06-14T22:30:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-06-14T22:30:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-06-14T22:30:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-06-14T22:30:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-06-14T22:30:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-06-14T22:30:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-06-14T22:30:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-06-14T22:30:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-06-14T22:30:31.334 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-06-14T22:30:31.615 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-06-14T22:30:31.639 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1008.ge47ff 12/12 2024-06-14T22:30:31.712 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-06-14T22:30:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:31.713 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-06-14T22:30:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1008.ge47ff4b8.el9.noarch 2024-06-14T22:30:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-14T22:30:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-14T22:30:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-14T22:30:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-14T22:30:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-14T22:30:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-14T22:30:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-14T22:30:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-14T22:30:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-14T22:30:31.713 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-14T22:30:31.714 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-14T22:30:31.714 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:31.714 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:32.071 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr-rook 2024-06-14T22:30:32.680 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:17 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:32.813 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:32.814 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:32.814 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-06-14T22:30:32.815 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:32.815 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-06-14T22:30:32.815 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook noarch 2:17.2.7-1008.ge47ff4b8.el9 ceph-noarch 47 k 2024-06-14T22:30:32.815 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-06-14T22:30:32.815 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-14T22:30:32.815 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-14T22:30:32.815 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-14T22:30:32.815 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-14T22:30:32.815 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-14T22:30:32.815 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-14T22:30:32.815 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-14T22:30:32.816 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-14T22:30:32.816 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-14T22:30:32.816 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-14T22:30:32.816 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-14T22:30:32.816 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-14T22:30:32.816 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:32.816 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-06-14T22:30:32.816 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:32.816 INFO:teuthology.orchestra.run.smithi046.stdout:Install 13 Packages 2024-06-14T22:30:32.816 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:32.817 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 2.2 M 2024-06-14T22:30:32.817 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 26 M 2024-06-14T22:30:32.817 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-06-14T22:30:33.331 INFO:teuthology.orchestra.run.smithi046.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 130 kB/s | 19 kB 00:00 2024-06-14T22:30:33.356 INFO:teuthology.orchestra.run.smithi046.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 149 kB/s | 26 kB 00:00 2024-06-14T22:30:33.531 INFO:teuthology.orchestra.run.smithi046.stdout:(3/13): ceph-mgr-rook-17.2.7-1008.ge47ff4b8.el9 136 kB/s | 47 kB 00:00 2024-06-14T22:30:33.557 INFO:teuthology.orchestra.run.smithi046.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 988 kB/s | 222 kB 00:00 2024-06-14T22:30:33.598 INFO:teuthology.orchestra.run.smithi046.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 732 kB/s | 159 kB 00:00 2024-06-14T22:30:33.690 INFO:teuthology.orchestra.run.smithi046.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 500 kB/s | 32 kB 00:00 2024-06-14T22:30:33.707 INFO:teuthology.orchestra.run.smithi046.stdout:(7/13): python3-certifi-2021.10.8-2.el9.noarch. 905 kB/s | 15 kB 00:00 2024-06-14T22:30:33.733 INFO:teuthology.orchestra.run.smithi046.stdout:(8/13): python3-google-auth-2.29.0-2.el9.noarch 8.1 MB/s | 205 kB 00:00 2024-06-14T22:30:33.750 INFO:teuthology.orchestra.run.smithi046.stdout:(9/13): python3-requests-oauthlib-1.3.0-12.el9. 321 kB/s | 54 kB 00:00 2024-06-14T22:30:33.775 INFO:teuthology.orchestra.run.smithi046.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.1 MB/s | 279 kB 00:00 2024-06-14T22:30:33.859 INFO:teuthology.orchestra.run.smithi046.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 545 kB/s | 59 kB 00:00 2024-06-14T22:30:33.883 INFO:teuthology.orchestra.run.smithi046.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.0 MB/s | 90 kB 00:00 2024-06-14T22:30:33.917 INFO:teuthology.orchestra.run.smithi046.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.6 MB/s | 1.0 MB 00:00 2024-06-14T22:30:33.920 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:33.921 INFO:teuthology.orchestra.run.smithi046.stdout:Total 2.0 MB/s | 2.2 MB 00:01 2024-06-14T22:30:33.987 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-06-14T22:30:34.003 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-06-14T22:30:34.003 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-06-14T22:30:34.086 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1008.ge47ff 12/12 2024-06-14T22:30:34.086 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1008.ge47ff 1/12 2024-06-14T22:30:34.086 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-06-14T22:30:34.087 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-06-14T22:30:34.087 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-06-14T22:30:34.087 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-06-14T22:30:34.087 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-06-14T22:30:34.087 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-06-14T22:30:34.087 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-06-14T22:30:34.087 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-06-14T22:30:34.087 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-06-14T22:30:34.087 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-06-14T22:30:34.099 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-06-14T22:30:34.099 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-06-14T22:30:34.530 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-06-14T22:30:34.530 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:34.530 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-06-14T22:30:34.530 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1008.ge47ff4b8.el9.noarch 2024-06-14T22:30:34.530 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-14T22:30:34.530 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-14T22:30:34.530 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-14T22:30:34.531 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-14T22:30:34.531 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-14T22:30:34.531 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-14T22:30:34.531 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-14T22:30:34.531 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-14T22:30:34.531 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-14T22:30:34.531 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-14T22:30:34.531 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-14T22:30:34.531 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:34.531 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:30:34.738 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-06-14T22:30:34.915 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-06-14T22:30:34.946 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-rook 2024-06-14T22:30:34.979 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-06-14T22:30:35.043 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-06-14T22:30:35.079 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-06-14T22:30:35.121 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-06-14T22:30:35.228 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-06-14T22:30:35.346 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 7/13 2024-06-14T22:30:35.414 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-06-14T22:30:35.534 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:18 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:30:35.660 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:30:35.661 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:35.661 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-06-14T22:30:35.661 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:35.661 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-06-14T22:30:35.661 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook noarch 2:17.2.7-1008.ge47ff4b8.el9 ceph-noarch 47 k 2024-06-14T22:30:35.661 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-06-14T22:30:35.662 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-14T22:30:35.662 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-14T22:30:35.662 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth noarch 1:2.29.0-2.el9 epel 205 k 2024-06-14T22:30:35.662 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-14T22:30:35.662 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-14T22:30:35.662 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-14T22:30:35.662 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-14T22:30:35.662 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-14T22:30:35.662 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-14T22:30:35.662 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-14T22:30:35.662 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-14T22:30:35.662 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-14T22:30:35.662 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:35.663 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-06-14T22:30:35.663 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:35.663 INFO:teuthology.orchestra.run.smithi178.stdout:Install 13 Packages 2024-06-14T22:30:35.663 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:35.663 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 2.2 M 2024-06-14T22:30:35.663 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 26 M 2024-06-14T22:30:35.663 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-06-14T22:30:36.144 INFO:teuthology.orchestra.run.smithi178.stdout:(1/13): ceph-mgr-rook-17.2.7-1008.ge47ff4b8.el9 236 kB/s | 47 kB 00:00 2024-06-14T22:30:36.181 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-06-14T22:30:36.203 INFO:teuthology.orchestra.run.smithi178.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 98 kB/s | 26 kB 00:00 2024-06-14T22:30:36.227 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-06-14T22:30:36.228 INFO:teuthology.orchestra.run.smithi178.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 67 kB/s | 19 kB 00:00 2024-06-14T22:30:36.261 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-06-14T22:30:36.307 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-06-14T22:30:36.330 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-rook-2:17.2.7-1008.ge47ff4b8.el9.noarch 13/13 2024-06-14T22:30:36.545 INFO:teuthology.orchestra.run.smithi178.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 464 kB/s | 159 kB 00:00 2024-06-14T22:30:36.653 INFO:teuthology.orchestra.run.smithi178.stdout:(5/13): python3-requests-oauthlib-1.3.0-12.el9. 495 kB/s | 54 kB 00:00 2024-06-14T22:30:36.687 INFO:teuthology.orchestra.run.smithi178.stdout:(6/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 409 kB/s | 222 kB 00:00 2024-06-14T22:30:36.712 INFO:teuthology.orchestra.run.smithi178.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 576 kB/s | 279 kB 00:00 2024-06-14T22:30:36.804 INFO:teuthology.orchestra.run.smithi178.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 214 kB/s | 32 kB 00:00 2024-06-14T22:30:36.821 INFO:teuthology.orchestra.run.smithi178.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 112 kB/s | 15 kB 00:00 2024-06-14T22:30:36.846 INFO:teuthology.orchestra.run.smithi178.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-06-14T22:30:36.872 INFO:teuthology.orchestra.run.smithi178.stdout:(11/13): python3-google-auth-2.29.0-2.el9.noarc 1.8 MB/s | 205 kB 00:00 2024-06-14T22:30:36.897 INFO:teuthology.orchestra.run.smithi178.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-06-14T22:30:36.939 INFO:teuthology.orchestra.run.smithi178.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.8 MB/s | 1.0 MB 00:00 2024-06-14T22:30:36.941 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:36.941 INFO:teuthology.orchestra.run.smithi178.stdout:Total 1.7 MB/s | 2.2 MB 00:01 2024-06-14T22:30:37.006 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-06-14T22:30:37.021 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-06-14T22:30:37.021 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-06-14T22:30:37.106 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-06-14T22:30:37.106 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-06-14T22:30:37.117 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1008.ge47ff4b8.el9.noarch 13/13 2024-06-14T22:30:37.117 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1008.ge47ff4b8.el9.noarch 1/13 2024-06-14T22:30:37.117 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-06-14T22:30:37.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-06-14T22:30:37.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-06-14T22:30:37.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-06-14T22:30:37.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-06-14T22:30:37.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-06-14T22:30:37.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-06-14T22:30:37.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-06-14T22:30:37.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 10/13 2024-06-14T22:30:37.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-06-14T22:30:37.118 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-06-14T22:30:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-06-14T22:30:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:37.453 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-06-14T22:30:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook-2:17.2.7-1008.ge47ff4b8.el9.noarch 2024-06-14T22:30:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-14T22:30:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-14T22:30:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-14T22:30:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-14T22:30:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-14T22:30:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-14T22:30:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-14T22:30:37.453 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-14T22:30:37.454 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-14T22:30:37.454 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-14T22:30:37.454 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-14T22:30:37.454 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-14T22:30:37.454 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:37.454 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:37.608 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-06-14T22:30:37.703 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-mgr-cephadm 2024-06-14T22:30:37.767 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-06-14T22:30:37.822 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-06-14T22:30:37.886 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-06-14T22:30:37.924 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-06-14T22:30:37.965 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-06-14T22:30:38.058 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-06-14T22:30:38.159 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-google-auth-1:2.29.0-2.el9.noarch 7/13 2024-06-14T22:30:38.208 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-06-14T22:30:38.306 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:23 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:38.438 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:38.439 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:38.439 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-06-14T22:30:38.440 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:38.440 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-06-14T22:30:38.440 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1008.ge47ff4b8.el9 ceph-noarch 123 k 2024-06-14T22:30:38.440 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-06-14T22:30:38.440 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-14T22:30:38.440 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-14T22:30:38.440 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-14T22:30:38.440 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-14T22:30:38.440 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-14T22:30:38.440 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:38.440 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-06-14T22:30:38.441 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:38.441 INFO:teuthology.orchestra.run.smithi046.stdout:Install 6 Packages 2024-06-14T22:30:38.441 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:38.441 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 7.0 M 2024-06-14T22:30:38.441 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 33 M 2024-06-14T22:30:38.441 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-06-14T22:30:38.914 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-06-14T22:30:38.937 INFO:teuthology.orchestra.run.smithi046.stdout:(1/6): ceph-mgr-cephadm-17.2.7-1008.ge47ff4b8.e 341 kB/s | 123 kB 00:00 2024-06-14T22:30:38.956 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-06-14T22:30:38.990 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-06-14T22:30:39.013 INFO:teuthology.orchestra.run.smithi046.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.3 MB/s | 548 kB 00:00 2024-06-14T22:30:39.035 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-06-14T22:30:39.038 INFO:teuthology.orchestra.run.smithi046.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-06-14T22:30:39.059 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-rook-2:17.2.7-1008.ge47ff4b8.el9.noarch 13/13 2024-06-14T22:30:39.063 INFO:teuthology.orchestra.run.smithi046.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 2.0 MB/s | 51 kB 00:00 2024-06-14T22:30:39.463 INFO:teuthology.orchestra.run.smithi046.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 279 kB/s | 248 kB 00:00 2024-06-14T22:30:39.880 INFO:teuthology.orchestra.run.smithi046.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.6 MB/s | 6.0 MB 00:01 2024-06-14T22:30:39.883 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:39.883 INFO:teuthology.orchestra.run.smithi046.stdout:Total 4.8 MB/s | 7.0 MB 00:01 2024-06-14T22:30:39.899 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1008.ge47ff4b8.el9.noarch 13/13 2024-06-14T22:30:39.899 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1008.ge47ff4b8.el9.noarch 1/13 2024-06-14T22:30:39.899 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-06-14T22:30:39.899 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-06-14T22:30:39.899 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-06-14T22:30:39.899 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-06-14T22:30:39.899 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-06-14T22:30:39.899 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-06-14T22:30:39.899 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-06-14T22:30:39.899 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-06-14T22:30:39.900 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-google-auth-1:2.29.0-2.el9.noarch 10/13 2024-06-14T22:30:39.900 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-06-14T22:30:39.900 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-06-14T22:30:39.959 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-06-14T22:30:39.971 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-06-14T22:30:39.971 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-06-14T22:30:40.075 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-06-14T22:30:40.075 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-06-14T22:30:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-06-14T22:30:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:40.233 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-06-14T22:30:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook-2:17.2.7-1008.ge47ff4b8.el9.noarch 2024-06-14T22:30:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-14T22:30:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-14T22:30:40.233 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth-1:2.29.0-2.el9.noarch 2024-06-14T22:30:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-14T22:30:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-14T22:30:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-14T22:30:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-14T22:30:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-14T22:30:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-14T22:30:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-14T22:30:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-14T22:30:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-14T22:30:40.234 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:40.234 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:30:40.390 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-06-14T22:30:40.486 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-cephadm 2024-06-14T22:30:40.572 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-06-14T22:30:40.625 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-06-14T22:30:41.060 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:24 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:30:41.187 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:30:41.187 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:41.187 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-06-14T22:30:41.187 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:41.188 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-06-14T22:30:41.188 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1008.ge47ff4b8.el9 ceph-noarch 123 k 2024-06-14T22:30:41.188 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-06-14T22:30:41.188 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-14T22:30:41.188 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-14T22:30:41.188 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-14T22:30:41.188 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-14T22:30:41.188 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-14T22:30:41.188 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:41.188 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-06-14T22:30:41.188 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:41.188 INFO:teuthology.orchestra.run.smithi178.stdout:Install 6 Packages 2024-06-14T22:30:41.188 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:41.189 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 7.0 M 2024-06-14T22:30:41.189 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 33 M 2024-06-14T22:30:41.189 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-06-14T22:30:41.222 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-06-14T22:30:41.313 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-06-14T22:30:41.383 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-06-14T22:30:41.419 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1008.ge47ff4b8.el9.noarch 6/6 2024-06-14T22:30:42.115 INFO:teuthology.orchestra.run.smithi178.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-06-14T22:30:42.141 INFO:teuthology.orchestra.run.smithi178.stdout:(2/6): ceph-mgr-cephadm-17.2.7-1008.ge47ff4b8.e 490 kB/s | 123 kB 00:00 2024-06-14T22:30:42.205 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1008.ge47ff4b8.el9.noarch 6/6 2024-06-14T22:30:42.205 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1008.ge47ff4b8.el9.noarch 1/6 2024-06-14T22:30:42.205 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-06-14T22:30:42.205 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-06-14T22:30:42.205 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-06-14T22:30:42.206 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-06-14T22:30:42.232 INFO:teuthology.orchestra.run.smithi178.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 4.6 MB/s | 548 kB 00:00 2024-06-14T22:30:42.258 INFO:teuthology.orchestra.run.smithi178.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 494 kB/s | 58 kB 00:00 2024-06-14T22:30:42.283 INFO:teuthology.orchestra.run.smithi178.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 1.0 MB/s | 51 kB 00:00 2024-06-14T22:30:42.476 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-06-14T22:30:42.476 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:42.476 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-06-14T22:30:42.476 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm-2:17.2.7-1008.ge47ff4b8.el9.noarch 2024-06-14T22:30:42.476 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-14T22:30:42.477 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-14T22:30:42.477 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-14T22:30:42.477 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-14T22:30:42.477 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-14T22:30:42.477 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:42.477 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:42.721 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-fuse 2024-06-14T22:30:43.319 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:28 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:43.450 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:43.451 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-06-14T22:30:43.451 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:43.451 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-06-14T22:30:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 812 k 2024-06-14T22:30:43.451 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-06-14T22:30:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-14T22:30:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:43.452 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-06-14T22:30:43.452 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:43.452 INFO:teuthology.orchestra.run.smithi046.stdout:Install 2 Packages 2024-06-14T22:30:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:43.452 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 892 k 2024-06-14T22:30:43.453 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 2.6 M 2024-06-14T22:30:43.453 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-06-14T22:30:43.741 INFO:teuthology.orchestra.run.smithi178.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.2 MB/s | 6.0 MB 00:01 2024-06-14T22:30:43.744 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:43.744 INFO:teuthology.orchestra.run.smithi178.stdout:Total 2.7 MB/s | 7.0 MB 00:02 2024-06-14T22:30:43.807 INFO:teuthology.orchestra.run.smithi046.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 390 kB/s | 80 kB 00:00 2024-06-14T22:30:43.815 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-06-14T22:30:43.826 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-06-14T22:30:43.826 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-06-14T22:30:43.924 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-06-14T22:30:43.924 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-06-14T22:30:44.024 INFO:teuthology.orchestra.run.smithi046.stdout:(2/2): ceph-fuse-17.2.7-1008.ge47ff4b8.el9.x86_ 1.9 MB/s | 812 kB 00:00 2024-06-14T22:30:44.024 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:44.025 INFO:teuthology.orchestra.run.smithi046.stdout:Total 1.5 MB/s | 892 kB 00:00 2024-06-14T22:30:44.037 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-06-14T22:30:44.060 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-06-14T22:30:44.060 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-06-14T22:30:44.142 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-06-14T22:30:44.143 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-06-14T22:30:44.231 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-06-14T22:30:44.344 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-06-14T22:30:44.407 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-06-14T22:30:44.465 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-06-14T22:30:44.468 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-06-14T22:30:44.543 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-fuse-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2/2 2024-06-14T22:30:44.985 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-06-14T22:30:45.071 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-06-14T22:30:45.127 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-06-14T22:30:45.155 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1008.ge47ff4b8.el9.noarch 6/6 2024-06-14T22:30:45.363 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2/2 2024-06-14T22:30:45.363 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-fuse-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/2 2024-06-14T22:30:45.643 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-06-14T22:30:45.643 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:45.643 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-06-14T22:30:45.643 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse-2:17.2.7-1008.ge47ff4b8.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-06-14T22:30:45.643 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:45.643 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:45.900 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1008.ge47ff4b8.el9.noarch 6/6 2024-06-14T22:30:45.900 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1008.ge47ff4b8.el9.noarch 1/6 2024-06-14T22:30:45.900 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-06-14T22:30:45.900 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-06-14T22:30:45.901 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-06-14T22:30:45.901 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-06-14T22:30:45.908 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install librados-devel 2024-06-14T22:30:46.166 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-06-14T22:30:46.166 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:46.166 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-06-14T22:30:46.166 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm-2:17.2.7-1008.ge47ff4b8.el9.noarch 2024-06-14T22:30:46.166 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-14T22:30:46.166 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-14T22:30:46.166 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-14T22:30:46.166 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-14T22:30:46.166 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-14T22:30:46.167 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:46.167 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:30:46.430 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-fuse 2024-06-14T22:30:46.511 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:31 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:46.642 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:46.643 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:46.643 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repo Size 2024-06-14T22:30:46.644 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:46.644 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-06-14T22:30:46.644 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 126 k 2024-06-14T22:30:46.644 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:46.644 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-06-14T22:30:46.644 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:46.644 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-06-14T22:30:46.644 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:46.644 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 126 k 2024-06-14T22:30:46.644 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 448 k 2024-06-14T22:30:46.644 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-06-14T22:30:47.001 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:30 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:30:47.038 INFO:teuthology.orchestra.run.smithi046.stdout:librados-devel-17.2.7-1008.ge47ff4b8.el9.x86_64 321 kB/s | 126 kB 00:00 2024-06-14T22:30:47.038 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:47.039 INFO:teuthology.orchestra.run.smithi046.stdout:Total 320 kB/s | 126 kB 00:00 2024-06-14T22:30:47.039 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-06-14T22:30:47.045 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-06-14T22:30:47.045 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-06-14T22:30:47.110 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-06-14T22:30:47.111 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-06-14T22:30:47.127 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:30:47.127 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:47.127 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-06-14T22:30:47.128 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:47.128 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-06-14T22:30:47.128 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 812 k 2024-06-14T22:30:47.128 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-06-14T22:30:47.128 INFO:teuthology.orchestra.run.smithi178.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-14T22:30:47.128 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:47.128 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-06-14T22:30:47.128 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:47.128 INFO:teuthology.orchestra.run.smithi178.stdout:Install 2 Packages 2024-06-14T22:30:47.128 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:47.129 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 892 k 2024-06-14T22:30:47.129 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 2.6 M 2024-06-14T22:30:47.129 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-06-14T22:30:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-06-14T22:30:47.351 INFO:teuthology.orchestra.run.smithi178.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 740 kB/s | 80 kB 00:00 2024-06-14T22:30:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librados-devel-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:30:47.677 INFO:teuthology.orchestra.run.smithi178.stdout:(2/2): ceph-fuse-17.2.7-1008.ge47ff4b8.el9.x86_ 1.8 MB/s | 812 kB 00:00 2024-06-14T22:30:47.678 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:47.678 INFO:teuthology.orchestra.run.smithi178.stdout:Total 1.6 MB/s | 892 kB 00:00 2024-06-14T22:30:47.689 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-06-14T22:30:47.711 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-06-14T22:30:47.711 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-06-14T22:30:47.788 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-06-14T22:30:47.788 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-06-14T22:30:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados-devel-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:30:47.991 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-06-14T22:30:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados-devel-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:30:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:48.092 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-06-14T22:30:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:30:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:48.093 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:48.131 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-06-14T22:30:48.200 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-fuse-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2/2 2024-06-14T22:30:48.325 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install libcephfs2 2024-06-14T22:30:48.858 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2/2 2024-06-14T22:30:48.858 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-fuse-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/2 2024-06-14T22:30:48.933 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:33 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:49.022 INFO:teuthology.orchestra.run.smithi046.stdout:Package libcephfs2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 is already installed. 2024-06-14T22:30:49.066 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:49.067 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-06-14T22:30:49.067 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:49.126 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install libcephfs-devel 2024-06-14T22:30:49.133 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-06-14T22:30:49.133 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:49.133 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-06-14T22:30:49.133 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse-2:17.2.7-1008.ge47ff4b8.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-06-14T22:30:49.133 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:49.133 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:30:49.384 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install librados-devel 2024-06-14T22:30:49.724 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:34 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:49.855 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:49.856 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:49.856 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repo Size 2024-06-14T22:30:49.856 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:49.856 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-06-14T22:30:49.857 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 25 k 2024-06-14T22:30:49.857 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:49.857 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-06-14T22:30:49.857 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:49.857 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-06-14T22:30:49.857 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:49.858 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 25 k 2024-06-14T22:30:49.858 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 110 k 2024-06-14T22:30:49.858 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-06-14T22:30:49.967 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:32 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:30:50.025 INFO:teuthology.orchestra.run.smithi046.stdout:libcephfs-devel-17.2.7-1008.ge47ff4b8.el9.x86_6 148 kB/s | 25 kB 00:00 2024-06-14T22:30:50.026 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:50.026 INFO:teuthology.orchestra.run.smithi046.stdout:Total 147 kB/s | 25 kB 00:00 2024-06-14T22:30:50.026 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-06-14T22:30:50.030 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-06-14T22:30:50.030 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-06-14T22:30:50.031 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-06-14T22:30:50.031 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-06-14T22:30:50.097 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:30:50.098 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:50.098 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repo Size 2024-06-14T22:30:50.098 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:50.098 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-06-14T22:30:50.098 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 126 k 2024-06-14T22:30:50.098 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:50.098 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-06-14T22:30:50.098 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:50.099 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-06-14T22:30:50.099 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:50.099 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 126 k 2024-06-14T22:30:50.099 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 448 k 2024-06-14T22:30:50.099 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-06-14T22:30:50.115 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-06-14T22:30:50.226 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs-devel-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:30:50.331 INFO:teuthology.orchestra.run.smithi178.stdout:librados-devel-17.2.7-1008.ge47ff4b8.el9.x86_64 545 kB/s | 126 kB 00:00 2024-06-14T22:30:50.332 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:50.332 INFO:teuthology.orchestra.run.smithi178.stdout:Total 541 kB/s | 126 kB 00:00 2024-06-14T22:30:50.332 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-06-14T22:30:50.337 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-06-14T22:30:50.337 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-06-14T22:30:50.399 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-06-14T22:30:50.400 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-06-14T22:30:50.555 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-06-14T22:30:50.593 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:30:50.661 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librados-devel-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:30:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs-devel-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:30:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:50.808 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-06-14T22:30:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:30:50.808 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:50.808 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:51.001 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install librados2 2024-06-14T22:30:51.090 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados-devel-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:30:51.322 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados-devel-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:30:51.323 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:51.323 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-06-14T22:30:51.323 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:30:51.323 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:51.323 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:30:51.582 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install libcephfs2 2024-06-14T22:30:51.595 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:36 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:51.681 INFO:teuthology.orchestra.run.smithi046.stdout:Package librados2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 is already installed. 2024-06-14T22:30:51.725 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:51.726 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-06-14T22:30:51.726 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:51.783 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install librbd1 2024-06-14T22:30:52.176 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:35 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:30:52.262 INFO:teuthology.orchestra.run.smithi178.stdout:Package libcephfs2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 is already installed. 2024-06-14T22:30:52.307 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:30:52.309 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-06-14T22:30:52.309 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:30:52.375 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:37 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:52.387 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install libcephfs-devel 2024-06-14T22:30:52.461 INFO:teuthology.orchestra.run.smithi046.stdout:Package librbd1-2:17.2.7-1008.ge47ff4b8.el9.x86_64 is already installed. 2024-06-14T22:30:52.504 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:52.506 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-06-14T22:30:52.506 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:52.572 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install python3-rados 2024-06-14T22:30:52.963 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:35 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:30:53.088 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:30:53.089 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:53.089 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repo Size 2024-06-14T22:30:53.089 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:53.089 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-06-14T22:30:53.089 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 25 k 2024-06-14T22:30:53.089 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:53.089 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-06-14T22:30:53.089 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:53.090 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-06-14T22:30:53.090 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:53.090 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 25 k 2024-06-14T22:30:53.090 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 110 k 2024-06-14T22:30:53.090 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-06-14T22:30:53.177 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:38 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:53.264 INFO:teuthology.orchestra.run.smithi046.stdout:Package python3-rados-2:17.2.7-1008.ge47ff4b8.el9.x86_64 is already installed. 2024-06-14T22:30:53.309 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:53.310 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-06-14T22:30:53.310 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:53.343 INFO:teuthology.orchestra.run.smithi178.stdout:libcephfs-devel-17.2.7-1008.ge47ff4b8.el9.x86_6 99 kB/s | 25 kB 00:00 2024-06-14T22:30:53.343 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:53.344 INFO:teuthology.orchestra.run.smithi178.stdout:Total 98 kB/s | 25 kB 00:00 2024-06-14T22:30:53.344 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-06-14T22:30:53.347 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-06-14T22:30:53.347 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-06-14T22:30:53.348 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-06-14T22:30:53.349 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-06-14T22:30:53.370 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install python3-rgw 2024-06-14T22:30:53.427 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-06-14T22:30:53.539 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs-devel-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:30:53.904 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:30:53.975 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:38 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:54.063 INFO:teuthology.orchestra.run.smithi046.stdout:Package python3-rgw-2:17.2.7-1008.ge47ff4b8.el9.x86_64 is already installed. 2024-06-14T22:30:54.107 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:54.108 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-06-14T22:30:54.109 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:54.120 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs-devel-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:30:54.120 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:54.120 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-06-14T22:30:54.120 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:30:54.120 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:54.120 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:30:54.167 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install python3-cephfs 2024-06-14T22:30:54.299 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install librados2 2024-06-14T22:30:54.769 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:39 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:54.855 INFO:teuthology.orchestra.run.smithi046.stdout:Package python3-cephfs-2:17.2.7-1008.ge47ff4b8.el9.x86_64 is already installed. 2024-06-14T22:30:54.874 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:37 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:30:54.899 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:54.900 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-06-14T22:30:54.900 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:54.957 INFO:teuthology.orchestra.run.smithi178.stdout:Package librados2-2:17.2.7-1008.ge47ff4b8.el9.x86_64 is already installed. 2024-06-14T22:30:54.962 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install python3-rbd 2024-06-14T22:30:54.999 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:30:55.000 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-06-14T22:30:55.000 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:30:55.085 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install librbd1 2024-06-14T22:30:55.566 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:40 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:55.655 INFO:teuthology.orchestra.run.smithi046.stdout:Package python3-rbd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 is already installed. 2024-06-14T22:30:55.660 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:38 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:30:55.701 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:55.702 INFO:teuthology.orchestra.run.smithi046.stdout:Nothing to do. 2024-06-14T22:30:55.702 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:55.743 INFO:teuthology.orchestra.run.smithi178.stdout:Package librbd1-2:17.2.7-1008.ge47ff4b8.el9.x86_64 is already installed. 2024-06-14T22:30:55.769 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install rbd-fuse 2024-06-14T22:30:55.787 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:30:55.788 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-06-14T22:30:55.788 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:30:55.858 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-rados 2024-06-14T22:30:56.365 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:41 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:56.439 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:39 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:30:56.495 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:56.496 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:56.496 INFO:teuthology.orchestra.run.smithi046.stdout: Package Architecture Version Repository Size 2024-06-14T22:30:56.496 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:56.496 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-06-14T22:30:56.496 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 82 k 2024-06-14T22:30:56.496 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:56.496 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-06-14T22:30:56.496 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:56.496 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-06-14T22:30:56.496 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:56.497 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 82 k 2024-06-14T22:30:56.497 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 222 k 2024-06-14T22:30:56.497 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-06-14T22:30:56.525 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-rados-2:17.2.7-1008.ge47ff4b8.el9.x86_64 is already installed. 2024-06-14T22:30:56.567 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:30:56.568 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-06-14T22:30:56.568 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:30:56.647 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-rgw 2024-06-14T22:30:56.820 INFO:teuthology.orchestra.run.smithi046.stdout:rbd-fuse-17.2.7-1008.ge47ff4b8.el9.x86_64.rpm 254 kB/s | 82 kB 00:00 2024-06-14T22:30:56.821 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:56.821 INFO:teuthology.orchestra.run.smithi046.stdout:Total 253 kB/s | 82 kB 00:00 2024-06-14T22:30:56.821 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-06-14T22:30:56.829 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-06-14T22:30:56.829 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-06-14T22:30:56.895 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-06-14T22:30:56.895 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-06-14T22:30:57.040 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-06-14T22:30:57.154 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-fuse-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:30:57.230 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:40 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:30:57.314 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-rgw-2:17.2.7-1008.ge47ff4b8.el9.x86_64 is already installed. 2024-06-14T22:30:57.357 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:30:57.358 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-06-14T22:30:57.358 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:30:57.435 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-cephfs 2024-06-14T22:30:57.518 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:30:57.757 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-fuse-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:30:57.757 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:57.757 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-06-14T22:30:57.757 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:30:57.757 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:57.757 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:30:58.010 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:41 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:30:58.022 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install rbd-mirror 2024-06-14T22:30:58.094 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-cephfs-2:17.2.7-1008.ge47ff4b8.el9.x86_64 is already installed. 2024-06-14T22:30:58.136 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:30:58.137 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-06-14T22:30:58.137 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:30:58.222 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-rbd 2024-06-14T22:30:58.628 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:43 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:30:58.761 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:30:58.762 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:58.762 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repo Size 2024-06-14T22:30:58.762 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:58.762 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-06-14T22:30:58.762 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 2.9 M 2024-06-14T22:30:58.762 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:58.762 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-06-14T22:30:58.762 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:30:58.762 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-06-14T22:30:58.762 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:30:58.762 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 2.9 M 2024-06-14T22:30:58.763 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 12 M 2024-06-14T22:30:58.763 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-06-14T22:30:58.801 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:41 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:30:58.886 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-rbd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 is already installed. 2024-06-14T22:30:58.928 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:30:58.929 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-06-14T22:30:58.929 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:30:59.010 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install rbd-fuse 2024-06-14T22:30:59.391 INFO:teuthology.orchestra.run.smithi046.stdout:rbd-mirror-17.2.7-1008.ge47ff4b8.el9.x86_64.rpm 4.7 MB/s | 2.9 MB 00:00 2024-06-14T22:30:59.391 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:59.392 INFO:teuthology.orchestra.run.smithi046.stdout:Total 4.7 MB/s | 2.9 MB 00:00 2024-06-14T22:30:59.392 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-06-14T22:30:59.402 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-06-14T22:30:59.402 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-06-14T22:30:59.488 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-06-14T22:30:59.489 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-06-14T22:30:59.585 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:42 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:30:59.710 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:30:59.710 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:59.710 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-06-14T22:30:59.711 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:59.711 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-06-14T22:30:59.711 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 82 k 2024-06-14T22:30:59.711 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:59.711 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-06-14T22:30:59.711 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:30:59.711 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-06-14T22:30:59.711 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:30:59.711 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 82 k 2024-06-14T22:30:59.711 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 222 k 2024-06-14T22:30:59.711 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-06-14T22:30:59.929 INFO:teuthology.orchestra.run.smithi178.stdout:rbd-fuse-17.2.7-1008.ge47ff4b8.el9.x86_64.rpm 377 kB/s | 82 kB 00:00 2024-06-14T22:30:59.929 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:30:59.929 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-06-14T22:30:59.930 INFO:teuthology.orchestra.run.smithi178.stdout:Total 375 kB/s | 82 kB 00:00 2024-06-14T22:30:59.930 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-06-14T22:30:59.936 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-06-14T22:30:59.936 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-06-14T22:30:59.983 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-mirror-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:30:59.997 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-06-14T22:30:59.998 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-06-14T22:31:00.011 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:31:00.011 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-14T22:31:00.011 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-14T22:31:00.011 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-14T22:31:00.011 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-14T22:31:00.011 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:31:00.142 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-06-14T22:31:00.281 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-fuse-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:31:00.625 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:31:00.872 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-fuse-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:31:00.872 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:31:00.872 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-06-14T22:31:00.872 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:31:00.872 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:31:00.872 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:31:01.140 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install rbd-mirror 2024-06-14T22:31:01.174 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-mirror-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:31:01.175 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:31:01.175 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-06-14T22:31:01.175 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:31:01.175 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:31:01.175 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:31:01.433 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install rbd-nbd 2024-06-14T22:31:01.734 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:44 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:31:01.865 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:31:01.866 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:31:01.866 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repo Size 2024-06-14T22:31:01.866 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:31:01.866 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-06-14T22:31:01.866 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 2.9 M 2024-06-14T22:31:01.866 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:31:01.866 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-06-14T22:31:01.867 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:31:01.867 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-06-14T22:31:01.867 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:31:01.867 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 2.9 M 2024-06-14T22:31:01.867 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 12 M 2024-06-14T22:31:01.867 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-06-14T22:31:02.034 INFO:teuthology.orchestra.run.smithi046.stdout:Last metadata expiration check: 0:01:47 ago on Fri 14 Jun 2024 10:29:15 PM UTC. 2024-06-14T22:31:02.165 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-06-14T22:31:02.165 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:31:02.166 INFO:teuthology.orchestra.run.smithi046.stdout: Package Architecture Version Repository Size 2024-06-14T22:31:02.166 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:31:02.166 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-06-14T22:31:02.166 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 165 k 2024-06-14T22:31:02.166 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:31:02.166 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-06-14T22:31:02.166 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-06-14T22:31:02.166 INFO:teuthology.orchestra.run.smithi046.stdout:Install 1 Package 2024-06-14T22:31:02.166 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:31:02.166 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 165 k 2024-06-14T22:31:02.166 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 477 k 2024-06-14T22:31:02.167 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-06-14T22:31:02.282 INFO:teuthology.orchestra.run.smithi178.stdout:rbd-mirror-17.2.7-1008.ge47ff4b8.el9.x86_64.rpm 7.1 MB/s | 2.9 MB 00:00 2024-06-14T22:31:02.283 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:31:02.283 INFO:teuthology.orchestra.run.smithi178.stdout:Total 7.0 MB/s | 2.9 MB 00:00 2024-06-14T22:31:02.283 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-06-14T22:31:02.293 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-06-14T22:31:02.293 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-06-14T22:31:02.374 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-06-14T22:31:02.374 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-06-14T22:31:02.517 INFO:teuthology.orchestra.run.smithi046.stdout:rbd-nbd-17.2.7-1008.ge47ff4b8.el9.x86_64.rpm 471 kB/s | 165 kB 00:00 2024-06-14T22:31:02.517 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:31:02.518 INFO:teuthology.orchestra.run.smithi046.stdout:Total 469 kB/s | 165 kB 00:00 2024-06-14T22:31:02.518 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-06-14T22:31:02.526 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-06-14T22:31:02.526 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-06-14T22:31:02.592 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-06-14T22:31:02.593 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-06-14T22:31:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-06-14T22:31:02.796 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-06-14T22:31:02.858 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-mirror-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:31:02.883 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-nbd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:31:02.885 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:31:02.885 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-14T22:31:02.885 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-14T22:31:02.885 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-14T22:31:02.886 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-14T22:31:02.886 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:31:03.296 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:31:03.538 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-nbd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:31:03.539 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:31:03.539 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-06-14T22:31:03.539 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:31:03.539 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:31:03.539 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-06-14T22:31:03.810 DEBUG:teuthology.parallel:result is None 2024-06-14T22:31:03.946 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-mirror-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:31:03.946 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:31:03.946 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-06-14T22:31:03.946 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:31:03.946 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:31:03.947 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:31:04.186 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install rbd-nbd 2024-06-14T22:31:04.780 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:47 ago on Fri 14 Jun 2024 10:29:17 PM UTC. 2024-06-14T22:31:04.906 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-06-14T22:31:04.907 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:31:04.907 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-06-14T22:31:04.907 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:31:04.907 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-06-14T22:31:04.907 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd x86_64 2:17.2.7-1008.ge47ff4b8.el9 ceph 165 k 2024-06-14T22:31:04.907 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:31:04.908 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-06-14T22:31:04.908 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-06-14T22:31:04.908 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-06-14T22:31:04.908 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:31:04.908 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 165 k 2024-06-14T22:31:04.908 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 477 k 2024-06-14T22:31:04.908 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-06-14T22:31:05.132 INFO:teuthology.orchestra.run.smithi178.stdout:rbd-nbd-17.2.7-1008.ge47ff4b8.el9.x86_64.rpm 737 kB/s | 165 kB 00:00 2024-06-14T22:31:05.132 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:31:05.132 INFO:teuthology.orchestra.run.smithi178.stdout:Total 733 kB/s | 165 kB 00:00 2024-06-14T22:31:05.133 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-06-14T22:31:05.140 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-06-14T22:31:05.141 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-06-14T22:31:05.203 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-06-14T22:31:05.203 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-06-14T22:31:05.357 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-06-14T22:31:05.462 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-nbd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:31:05.889 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:31:06.156 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-nbd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 1/1 2024-06-14T22:31:06.156 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:31:06.156 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-06-14T22:31:06.156 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd-2:17.2.7-1008.ge47ff4b8.el9.x86_64 2024-06-14T22:31:06.156 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:31:06.156 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-06-14T22:31:06.410 DEBUG:teuthology.parallel:result is None 2024-06-14T22:31:06.410 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-14T22:31:06.410 INFO:teuthology.packaging:ref: None 2024-06-14T22:31:06.411 INFO:teuthology.packaging:tag: None 2024-06-14T22:31:06.411 INFO:teuthology.packaging:branch: quincy 2024-06-14T22:31:06.411 INFO:teuthology.packaging:sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-14T22:31:06.411 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-06-14T22:31:06.607 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-14T22:31:06.634 INFO:teuthology.orchestra.run.smithi046.stdout:17.2.7-1008.ge47ff4b8.el9 2024-06-14T22:31:06.634 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1008.ge47ff4b8.el9 2024-06-14T22:31:06.635 INFO:teuthology.task.install:The correct ceph version 17.2.7-1008.ge47ff4b8 is installed. 2024-06-14T22:31:06.636 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-14T22:31:06.636 INFO:teuthology.packaging:ref: None 2024-06-14T22:31:06.636 INFO:teuthology.packaging:tag: None 2024-06-14T22:31:06.636 INFO:teuthology.packaging:branch: quincy 2024-06-14T22:31:06.637 INFO:teuthology.packaging:sha1: 6f4b09560ed399c3f139402a02f389c2fc8bb2cc 2024-06-14T22:31:06.637 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=quincy 2024-06-14T22:31:06.831 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-14T22:31:06.853 INFO:teuthology.orchestra.run.smithi178.stdout:17.2.7-1008.ge47ff4b8.el9 2024-06-14T22:31:06.854 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1008.ge47ff4b8.el9 2024-06-14T22:31:06.854 INFO:teuthology.task.install:The correct ceph version 17.2.7-1008.ge47ff4b8 is installed. 2024-06-14T22:31:06.855 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-14T22:31:06.855 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-06-14T22:31:06.856 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-14T22:31:06.887 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-14T22:31:06.887 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-14T22:31:06.921 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-14T22:31:06.921 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-06-14T22:31:06.921 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2024-06-14T22:31:06.957 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-14T22:31:07.026 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-14T22:31:07.026 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2024-06-14T22:31:07.052 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-14T22:31:07.127 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-14T22:31:07.128 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-06-14T22:31:07.128 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-14T22:31:07.156 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-14T22:31:07.224 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-14T22:31:07.225 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-14T22:31:07.249 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-14T22:31:07.314 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-14T22:31:07.314 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-06-14T22:31:07.315 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2024-06-14T22:31:07.343 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-14T22:31:07.411 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-14T22:31:07.411 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/stdin-killer 2024-06-14T22:31:07.435 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-14T22:31:07.499 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-14T22:31:07.581 INFO:tasks.cephadm:Config: {'cephadm_branch': 'quincy', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}, 'global': {'mon election default strategy': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'image': 'quay.ceph.io/ceph-ci/ceph:quincy', 'create_rbd_pool': True, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '6f4b09560ed399c3f139402a02f389c2fc8bb2cc'} 2024-06-14T22:31:07.581 INFO:tasks.cephadm:Cluster image is quay.ceph.io/ceph-ci/ceph:quincy 2024-06-14T22:31:07.582 INFO:tasks.cephadm:Cluster fsid is cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:31:07.582 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-14T22:31:07.582 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.46', 'mon.c': '[v2:172.21.15.46:3301,v1:172.21.15.46:6790]', 'mon.b': '172.21.15.178'} 2024-06-14T22:31:07.582 INFO:tasks.cephadm:First mon is mon.a on smithi046 2024-06-14T22:31:07.582 INFO:tasks.cephadm:First mgr is y 2024-06-14T22:31:07.583 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-14T22:31:07.583 DEBUG:teuthology.orchestra.run.smithi046:> sudo hostname $(hostname -s) 2024-06-14T22:31:07.611 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname $(hostname -s) 2024-06-14T22:31:07.636 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref quincy)... 2024-06-14T22:31:07.636 DEBUG:teuthology.orchestra.run.smithi046:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-14T22:31:07.831 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Jun 14 22:31 /home/ubuntu/cephtest/cephadm 2024-06-14T22:31:07.831 DEBUG:teuthology.orchestra.run.smithi178:> curl --silent https://raw.githubusercontent.com/ceph/ceph/quincy/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-14T22:31:08.018 INFO:teuthology.orchestra.run.smithi178.stdout:-rw-r--r--. 1 ubuntu ubuntu 375153 Jun 14 22:31 /home/ubuntu/cephtest/cephadm 2024-06-14T22:31:08.019 DEBUG:teuthology.orchestra.run.smithi046:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-14T22:31:08.038 DEBUG:teuthology.orchestra.run.smithi178:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-14T22:31:08.064 INFO:tasks.cephadm:Pulling image quay.ceph.io/ceph-ci/ceph:quincy on all hosts... 2024-06-14T22:31:08.064 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-06-14T22:31:08.081 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy pull 2024-06-14T22:31:08.950 INFO:teuthology.orchestra.run.smithi046.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-06-14T22:31:08.971 INFO:teuthology.orchestra.run.smithi178.stderr:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-06-14T22:31:34.906 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-06-14T22:31:34.906 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_version": "ceph version 17.2.7-1008-ge47ff4b8 (e47ff4b87f747658fc7376cc7065fc56801eb37b) quincy (stable)", 2024-06-14T22:31:34.906 INFO:teuthology.orchestra.run.smithi178.stdout: "image_id": "37f9eedc23c32ee6d04590e8a81233336d2c14cd278f52a2a2746a97e6ef2d95", 2024-06-14T22:31:34.906 INFO:teuthology.orchestra.run.smithi178.stdout: "repo_digests": [ 2024-06-14T22:31:34.906 INFO:teuthology.orchestra.run.smithi178.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580" 2024-06-14T22:31:34.906 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-06-14T22:31:34.906 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-06-14T22:31:41.167 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-06-14T22:31:41.167 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 17.2.7-1008-ge47ff4b8 (e47ff4b87f747658fc7376cc7065fc56801eb37b) quincy (stable)", 2024-06-14T22:31:41.167 INFO:teuthology.orchestra.run.smithi046.stdout: "image_id": "37f9eedc23c32ee6d04590e8a81233336d2c14cd278f52a2a2746a97e6ef2d95", 2024-06-14T22:31:41.167 INFO:teuthology.orchestra.run.smithi046.stdout: "repo_digests": [ 2024-06-14T22:31:41.167 INFO:teuthology.orchestra.run.smithi046.stdout: "quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580" 2024-06-14T22:31:41.167 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-06-14T22:31:41.167 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-06-14T22:31:41.187 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph 2024-06-14T22:31:41.228 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph 2024-06-14T22:31:41.262 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /etc/ceph 2024-06-14T22:31:41.295 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /etc/ceph 2024-06-14T22:31:41.330 INFO:tasks.cephadm:Writing seed config... 2024-06-14T22:31:41.331 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-06-14T22:31:41.331 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-06-14T22:31:41.332 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-14T22:31:41.332 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-14T22:31:41.332 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-06-14T22:31:41.332 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-06-14T22:31:41.332 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-14T22:31:41.332 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-14T22:31:41.332 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-14T22:31:41.332 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-14T22:31:41.332 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-14T22:31:41.333 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-06-14T22:31:41.333 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-14T22:31:41.351 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = cb009720-2a9d-11ef-bc9d-c7b262605968 mon election default strategy = 1 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-06-14T22:31:41.352 DEBUG:teuthology.orchestra.run.smithi046:mon.a> sudo journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mon.a.service 2024-06-14T22:31:41.394 DEBUG:teuthology.orchestra.run.smithi046:mgr.y> sudo journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mgr.y.service 2024-06-14T22:31:41.436 INFO:tasks.cephadm:Bootstrapping... 2024-06-14T22:31:41.437 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy -v bootstrap --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.46 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-14T22:31:41.649 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-06-14T22:31:41.649 INFO:teuthology.orchestra.run.smithi046.stdout:cephadm ['--image', 'quay.ceph.io/ceph-ci/ceph:quincy', '-v', 'bootstrap', '--fsid', 'cb009720-2a9d-11ef-bc9d-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.46', '--skip-admin-label'] 2024-06-14T22:31:41.675 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 5.1.0 2024-06-14T22:31:41.675 INFO:teuthology.orchestra.run.smithi046.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-14T22:31:41.676 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying podman|docker is present... 2024-06-14T22:31:41.700 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 5.1.0 2024-06-14T22:31:41.700 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying lvm2 is present... 2024-06-14T22:31:41.700 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying time synchronization is in place... 2024-06-14T22:31:41.708 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-14T22:31:41.709 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-14T22:31:41.715 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-14T22:31:41.715 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-06-14T22:31:41.722 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout enabled 2024-06-14T22:31:41.729 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout active 2024-06-14T22:31:41.729 INFO:teuthology.orchestra.run.smithi046.stdout:Unit chronyd.service is enabled and running 2024-06-14T22:31:41.729 INFO:teuthology.orchestra.run.smithi046.stdout:Repeating the final host check... 2024-06-14T22:31:41.753 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 5.1.0 2024-06-14T22:31:41.753 INFO:teuthology.orchestra.run.smithi046.stdout:podman (/bin/podman) version 5.1.0 is present 2024-06-14T22:31:41.753 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl is present 2024-06-14T22:31:41.753 INFO:teuthology.orchestra.run.smithi046.stdout:lvcreate is present 2024-06-14T22:31:41.760 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-14T22:31:41.760 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-14T22:31:41.767 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-14T22:31:41.767 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-06-14T22:31:41.774 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout enabled 2024-06-14T22:31:41.781 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout active 2024-06-14T22:31:41.781 INFO:teuthology.orchestra.run.smithi046.stdout:Unit chronyd.service is enabled and running 2024-06-14T22:31:41.781 INFO:teuthology.orchestra.run.smithi046.stdout:Host looks OK 2024-06-14T22:31:41.781 INFO:teuthology.orchestra.run.smithi046.stdout:Cluster fsid: cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:31:41.781 INFO:teuthology.orchestra.run.smithi046.stdout:Acquiring lock 140696257004448 on /run/cephadm/cb009720-2a9d-11ef-bc9d-c7b262605968.lock 2024-06-14T22:31:41.782 INFO:teuthology.orchestra.run.smithi046.stdout:Lock 140696257004448 acquired on /run/cephadm/cb009720-2a9d-11ef-bc9d-c7b262605968.lock 2024-06-14T22:31:41.782 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying IP 172.21.15.46 port 3300 ... 2024-06-14T22:31:41.782 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying IP 172.21.15.46 port 6789 ... 2024-06-14T22:31:41.782 INFO:teuthology.orchestra.run.smithi046.stdout:Base mon IP(s) is [172.21.15.46:3300, 172.21.15.46:6789], mon addrv is [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-06-14T22:31:41.786 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.46 metric 100 2024-06-14T22:31:41.786 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.46 metric 100 2024-06-14T22:31:41.789 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-14T22:31:41.789 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-06-14T22:31:41.789 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-06-14T22:31:41.791 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-14T22:31:41.792 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-14T22:31:41.792 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-14T22:31:41.792 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-06-14T22:31:41.792 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-06-14T22:31:41.792 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-14T22:31:41.792 INFO:teuthology.orchestra.run.smithi046.stdout:Mon IP `172.21.15.46` is in CIDR network `172.21.0.0/20` 2024-06-14T22:31:41.792 INFO:teuthology.orchestra.run.smithi046.stdout:Mon IP `172.21.15.46` is in CIDR network `172.21.0.0/20` 2024-06-14T22:31:41.793 INFO:teuthology.orchestra.run.smithi046.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-14T22:31:41.793 INFO:teuthology.orchestra.run.smithi046.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-14T22:31:41.793 INFO:teuthology.orchestra.run.smithi046.stdout:Pulling container image quay.ceph.io/ceph-ci/ceph:quincy... 2024-06-14T22:31:42.233 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 37f9eedc23c32ee6d04590e8a81233336d2c14cd278f52a2a2746a97e6ef2d95 2024-06-14T22:31:42.233 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Trying to pull quay.ceph.io/ceph-ci/ceph:quincy... 2024-06-14T22:31:42.233 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Getting image source signatures 2024-06-14T22:31:42.233 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying blob sha256:ae1bd4835d962ece7feffb132e988e26ca074ae51fc4b41b3d8d10a6caf0861f 2024-06-14T22:31:42.233 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying blob sha256:9065cd298533077f0006921bb12bd331d24f6c140cc8fa972c2e6c4e382038d1 2024-06-14T22:31:42.233 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying config sha256:37f9eedc23c32ee6d04590e8a81233336d2c14cd278f52a2a2746a97e6ef2d95 2024-06-14T22:31:42.233 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-14T22:31:43.222 INFO:teuthology.orchestra.run.smithi046.stdout:ceph: stdout ceph version 17.2.7-1008-ge47ff4b8 (e47ff4b87f747658fc7376cc7065fc56801eb37b) quincy (stable) 2024-06-14T22:31:43.222 INFO:teuthology.orchestra.run.smithi046.stdout:Ceph version: ceph version 17.2.7-1008-ge47ff4b8 (e47ff4b87f747658fc7376cc7065fc56801eb37b) quincy (stable) 2024-06-14T22:31:43.222 INFO:teuthology.orchestra.run.smithi046.stdout:Extracting ceph user uid/gid from container image... 2024-06-14T22:31:44.236 INFO:teuthology.orchestra.run.smithi046.stdout:stat: stdout 167 167 2024-06-14T22:31:44.236 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial keys... 2024-06-14T22:31:45.208 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQDQxGxmoQw9JRAAC3ulA4q+YVZuqqZOqC5Cog== 2024-06-14T22:31:46.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQDRxGxmJg8VIRAA0gO+AfszvDjdeNDMOSKj7Q== 2024-06-14T22:31:47.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQDSxGxm34/nJRAAYOWX0Ky4+G8aOc3ZOV9Kpg== 2024-06-14T22:31:47.186 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial monmap... 2024-06-14T22:31:48.274 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-14T22:31:48.275 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-06-14T22:31:48.275 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:31:48.275 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-14T22:31:48.275 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool for a [v2:172.21.15.46:3300,v1:172.21.15.46:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-14T22:31:48.275 INFO:teuthology.orchestra.run.smithi046.stdout:setting min_mon_release = octopus 2024-06-14T22:31:48.275 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: set fsid to cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:31:48.275 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-14T22:31:48.275 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:31:48.275 INFO:teuthology.orchestra.run.smithi046.stdout:Creating mon... 2024-06-14T22:31:49.338 INFO:teuthology.orchestra.run.smithi046.stdout:create mon.a on 2024-06-14T22:31:49.551 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-14T22:31:49.697 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-14T22:31:49.853 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-cb009720-2a9d-11ef-bc9d-c7b262605968.target → /etc/systemd/system/ceph-cb009720-2a9d-11ef-bc9d-c7b262605968.target. 2024-06-14T22:31:49.854 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-cb009720-2a9d-11ef-bc9d-c7b262605968.target → /etc/systemd/system/ceph-cb009720-2a9d-11ef-bc9d-c7b262605968.target. 2024-06-14T22:31:50.029 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mon.a 2024-06-14T22:31:50.029 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to reset failed state of unit ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mon.a.service: Unit ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mon.a.service not loaded. 2024-06-14T22:31:50.200 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-cb009720-2a9d-11ef-bc9d-c7b262605968.target.wants/ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mon.a.service → /etc/systemd/system/ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@.service. 2024-06-14T22:31:50.571 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:50 smithi046 podman[29075]: 2024-06-14 22:31:50.450113323 +0000 UTC m=+0.116676479 container create f0047fe4f2dbf913ecad288244faa73f047eb4eff0a10ede6c8ac655e90bf86f (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-a, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-e47ff4b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.build-date=20240612) 2024-06-14T22:31:50.786 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-14T22:31:50.786 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-06-14T22:31:50.794 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-14T22:31:50.795 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-06-14T22:31:50.795 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-06-14T22:31:50.795 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to enable service . firewalld.service is not available 2024-06-14T22:31:50.795 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mon to start... 2024-06-14T22:31:50.795 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mon... 2024-06-14T22:31:50.916 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:50 smithi046 podman[29075]: 2024-06-14 22:31:50.669550461 +0000 UTC m=+0.336113630 container init f0047fe4f2dbf913ecad288244faa73f047eb4eff0a10ede6c8ac655e90bf86f (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, ceph=True, org.label-schema.schema-version=1.0, RELEASE=quincy-e47ff4b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240612) 2024-06-14T22:31:50.916 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:50 smithi046 podman[29075]: 2024-06-14 22:31:50.675117948 +0000 UTC m=+0.341681103 container start f0047fe4f2dbf913ecad288244faa73f047eb4eff0a10ede6c8ac655e90bf86f (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240612, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, RELEASE=quincy-e47ff4b) 2024-06-14T22:31:50.916 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:50 smithi046 ceph-mon[29113]: mkfs cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:31:50.916 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:50 smithi046 ceph-mon[29113]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-14T22:31:50.916 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:50 smithi046 bash[29075]: f0047fe4f2dbf913ecad288244faa73f047eb4eff0a10ede6c8ac655e90bf86f 2024-06-14T22:31:50.916 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:50 smithi046 systemd[1]: Started Ceph mon.a for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:31:51.878 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:51 smithi046 ceph-mon[29113]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-14T22:31:51.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:51 smithi046 ceph-mon[29113]: monmap e1: 1 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-14T22:31:51.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:51 smithi046 ceph-mon[29113]: fsmap 2024-06-14T22:31:51.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:51 smithi046 ceph-mon[29113]: osdmap e1: 0 total, 0 up, 0 in 2024-06-14T22:31:51.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:51 smithi046 ceph-mon[29113]: mgrmap e1: no daemons active 2024-06-14T22:31:51.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:51 smithi046 ceph-mon[29113]: from='client.? 172.21.15.46:0/2591717670' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-14T22:31:51.883 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout cluster: 2024-06-14T22:31:51.883 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout id: cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:31:51.883 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-14T22:31:51.883 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-06-14T22:31:51.883 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout services: 2024-06-14T22:31:51.883 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.599225s) 2024-06-14T22:31:51.884 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-14T22:31:51.884 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-14T22:31:51.884 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-06-14T22:31:51.884 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout data: 2024-06-14T22:31:51.884 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-14T22:31:51.884 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-14T22:31:51.884 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-14T22:31:51.884 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout pgs: 2024-06-14T22:31:51.884 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-06-14T22:31:51.884 INFO:teuthology.orchestra.run.smithi046.stdout:mon is available 2024-06-14T22:31:51.884 INFO:teuthology.orchestra.run.smithi046.stdout:Assimilating anything we can from ceph.conf... 2024-06-14T22:31:52.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:52 smithi046 ceph-mon[29113]: from='client.? 172.21.15.46:0/926552516' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-14T22:31:52.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:52 smithi046 ceph-mon[29113]: from='client.? 172.21.15.46:0/926552516' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-14T22:31:52.964 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-06-14T22:31:52.964 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [global] 2024-06-14T22:31:52.964 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout fsid = cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:31:52.964 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-06-14T22:31:52.965 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-14T22:31:52.965 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-14T22:31:52.965 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-14T22:31:52.965 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-14T22:31:52.965 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-06-14T22:31:52.965 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-14T22:31:52.965 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-14T22:31:52.965 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-06-14T22:31:52.965 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [osd] 2024-06-14T22:31:52.965 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-14T22:31:52.965 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-14T22:31:52.965 INFO:teuthology.orchestra.run.smithi046.stdout:Generating new minimal ceph.conf... 2024-06-14T22:31:53.941 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:53 smithi046 ceph-mon[29113]: from='client.? 172.21.15.46:0/4135752194' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:31:53.960 INFO:teuthology.orchestra.run.smithi046.stdout:Restarting the monitor... 2024-06-14T22:31:54.231 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:53 smithi046 systemd[1]: Stopping Ceph mon.a for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:31:54.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:54 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-a[29090]: 2024-06-14T22:31:54.234+0000 7f708a7ef640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-14T22:31:54.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:54 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-a[29090]: 2024-06-14T22:31:54.234+0000 7f708a7ef640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-14T22:31:54.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:54 smithi046 podman[29430]: 2024-06-14 22:31:54.399481084 +0000 UTC m=+0.267144246 container died f0047fe4f2dbf913ecad288244faa73f047eb4eff0a10ede6c8ac655e90bf86f (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.build-date=20240612, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=quincy-e47ff4b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-14T22:31:55.128 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:54 smithi046 podman[29430]: 2024-06-14 22:31:54.89873532 +0000 UTC m=+0.766398481 container cleanup f0047fe4f2dbf913ecad288244faa73f047eb4eff0a10ede6c8ac655e90bf86f (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240612, RELEASE=quincy-e47ff4b, CEPH_POINT_RELEASE=, ceph=True) 2024-06-14T22:31:55.128 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:54 smithi046 bash[29430]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-a 2024-06-14T22:31:55.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 podman[29449]: 2024-06-14 22:31:55.133121103 +0000 UTC m=+0.728926115 container remove f0047fe4f2dbf913ecad288244faa73f047eb4eff0a10ede6c8ac655e90bf86f (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=quincy-e47ff4b, org.label-schema.build-date=20240612, io.buildah.version=1.36.0, GIT_BRANCH=HEAD) 2024-06-14T22:31:55.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-14T22:31:55.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 systemd[1]: Stopped Ceph mon.a for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:31:55.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 systemd[1]: Starting Ceph mon.a for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:31:55.764 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 podman[29548]: 2024-06-14 22:31:55.405577516 +0000 UTC m=+0.017982578 image pull 37f9eedc23c32ee6d04590e8a81233336d2c14cd278f52a2a2746a97e6ef2d95 quay.ceph.io/ceph-ci/ceph:quincy 2024-06-14T22:31:55.765 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 podman[29548]: 2024-06-14 22:31:55.57563124 +0000 UTC m=+0.188036291 container create 4ab3984515da7ca2360c97fdd9f950c9bb2ad086792cff640dc8fb9b6204b5b3 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240612, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, RELEASE=quincy-e47ff4b, GIT_CLEAN=True) 2024-06-14T22:31:55.765 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 podman[29548]: 2024-06-14 22:31:55.769741001 +0000 UTC m=+0.382146067 container init 4ab3984515da7ca2360c97fdd9f950c9bb2ad086792cff640dc8fb9b6204b5b3 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240612, io.buildah.version=1.36.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=quincy-e47ff4b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-14T22:31:55.852 INFO:teuthology.orchestra.run.smithi046.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-14T22:31:56.024 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 podman[29548]: 2024-06-14 22:31:55.775768106 +0000 UTC m=+0.388173158 container start 4ab3984515da7ca2360c97fdd9f950c9bb2ad086792cff640dc8fb9b6204b5b3 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240612, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=quincy-e47ff4b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-14T22:31:56.024 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: set uid:gid to 167:167 (ceph:ceph) 2024-06-14T22:31:56.024 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: ceph version 17.2.7-1008-ge47ff4b8 (e47ff4b87f747658fc7376cc7065fc56801eb37b) quincy (stable), process ceph-mon, pid 7 2024-06-14T22:31:56.024 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: pidfile_write: ignore empty --pid-file 2024-06-14T22:31:56.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: load: jerasure load: lrc 2024-06-14T22:31:56.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: RocksDB version: 6.15.5 2024-06-14T22:31:56.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-06-14T22:31:56.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Compile date Jun 14 2024 2024-06-14T22:31:56.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: DB SUMMARY 2024-06-14T22:31:56.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: DB Session ID: JYE5V9VNQDHAA8KM4SSU 2024-06-14T22:31:56.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: CURRENT file: CURRENT 2024-06-14T22:31:56.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: IDENTITY file: IDENTITY 2024-06-14T22:31:56.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-06-14T22:31:56.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-14T22:31:56.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 83714 ; 2024-06-14T22:31:56.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.error_if_exists: 0 2024-06-14T22:31:56.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.create_if_missing: 0 2024-06-14T22:31:56.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.paranoid_checks: 1 2024-06-14T22:31:56.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-14T22:31:56.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.env: 0x560c33d4c440 2024-06-14T22:31:56.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.fs: Posix File System 2024-06-14T22:31:56.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.info_log: 0x560c358c4f40 2024-06-14T22:31:56.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_file_opening_threads: 16 2024-06-14T22:31:56.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.statistics: (nil) 2024-06-14T22:31:56.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.use_fsync: 0 2024-06-14T22:31:56.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_log_file_size: 0 2024-06-14T22:31:56.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-14T22:31:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-14T22:31:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.keep_log_file_num: 1000 2024-06-14T22:31:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.recycle_log_file_num: 0 2024-06-14T22:31:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.allow_fallocate: 1 2024-06-14T22:31:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.allow_mmap_reads: 0 2024-06-14T22:31:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.allow_mmap_writes: 0 2024-06-14T22:31:56.027 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.use_direct_reads: 0 2024-06-14T22:31:56.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-14T22:31:56.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.create_missing_column_families: 0 2024-06-14T22:31:56.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.db_log_dir: 2024-06-14T22:31:56.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-06-14T22:31:56.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-14T22:31:56.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-14T22:31:56.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-14T22:31:56.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-14T22:31:56.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-14T22:31:56.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-14T22:31:56.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.advise_random_on_open: 1 2024-06-14T22:31:56.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.db_write_buffer_size: 0 2024-06-14T22:31:56.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.write_buffer_manager: 0x560c359827b0 2024-06-14T22:31:56.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-14T22:31:56.030 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-06-14T22:31:56.030 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-14T22:31:56.030 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-14T22:31:56.030 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.rate_limiter: (nil) 2024-06-14T22:31:56.030 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-14T22:31:56.030 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.wal_recovery_mode: 2 2024-06-14T22:31:56.030 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.enable_thread_tracking: 0 2024-06-14T22:31:56.030 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.enable_pipelined_write: 0 2024-06-14T22:31:56.030 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.unordered_write: 0 2024-06-14T22:31:56.030 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-14T22:31:56.030 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-14T22:31:56.030 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-14T22:31:56.031 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-14T22:31:56.031 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.row_cache: None 2024-06-14T22:31:56.031 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.wal_filter: None 2024-06-14T22:31:56.031 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-14T22:31:56.031 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.allow_ingest_behind: 0 2024-06-14T22:31:56.031 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.preserve_deletes: 0 2024-06-14T22:31:56.031 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.two_write_queues: 0 2024-06-14T22:31:56.031 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.manual_wal_flush: 0 2024-06-14T22:31:56.031 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.atomic_flush: 0 2024-06-14T22:31:56.031 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-14T22:31:56.031 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-14T22:31:56.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-14T22:31:56.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.log_readahead_size: 0 2024-06-14T22:31:56.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-14T22:31:56.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.best_efforts_recovery: 0 2024-06-14T22:31:56.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-14T22:31:56.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-14T22:31:56.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.allow_data_in_errors: 0 2024-06-14T22:31:56.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.db_host_id: __hostname__ 2024-06-14T22:31:56.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_background_jobs: 2 2024-06-14T22:31:56.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_background_compactions: -1 2024-06-14T22:31:56.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_subcompactions: 1 2024-06-14T22:31:56.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-14T22:31:56.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-14T22:31:56.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-14T22:31:56.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_total_wal_size: 0 2024-06-14T22:31:56.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-14T22:31:56.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-14T22:31:56.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-14T22:31:56.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-14T22:31:56.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_open_files: -1 2024-06-14T22:31:56.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.bytes_per_sync: 0 2024-06-14T22:31:56.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-14T22:31:56.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-14T22:31:56.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compaction_readahead_size: 0 2024-06-14T22:31:56.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_background_flushes: -1 2024-06-14T22:31:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Compression algorithms supported: 2024-06-14T22:31:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: kZSTD supported: 0 2024-06-14T22:31:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: kXpressCompression supported: 0 2024-06-14T22:31:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: kBZip2Compression supported: 0 2024-06-14T22:31:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-14T22:31:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: kLZ4Compression supported: 1 2024-06-14T22:31:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: kZlibCompression supported: 1 2024-06-14T22:31:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: kLZ4HCCompression supported: 1 2024-06-14T22:31:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: kSnappyCompression supported: 1 2024-06-14T22:31:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-14T22:31:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-06-14T22:31:56.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-06-14T22:31:56.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-14T22:31:56.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.merge_operator: 2024-06-14T22:31:56.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compaction_filter: None 2024-06-14T22:31:56.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compaction_filter_factory: None 2024-06-14T22:31:56.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.sst_partitioner_factory: None 2024-06-14T22:31:56.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-14T22:31:56.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-14T22:31:56.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560c3586ba68) 2024-06-14T22:31:56.035 INFO:journalctl@ceph.mon.a.smithi046.stdout: cache_index_and_filter_blocks: 1 2024-06-14T22:31:56.035 INFO:journalctl@ceph.mon.a.smithi046.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-14T22:31:56.035 INFO:journalctl@ceph.mon.a.smithi046.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-14T22:31:56.035 INFO:journalctl@ceph.mon.a.smithi046.stdout: pin_top_level_index_and_filter: 1 2024-06-14T22:31:56.035 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_type: 0 2024-06-14T22:31:56.036 INFO:journalctl@ceph.mon.a.smithi046.stdout: data_block_index_type: 0 2024-06-14T22:31:56.036 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_shortening: 1 2024-06-14T22:31:56.036 INFO:journalctl@ceph.mon.a.smithi046.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-14T22:31:56.036 INFO:journalctl@ceph.mon.a.smithi046.stdout: hash_index_allow_collision: 1 2024-06-14T22:31:56.036 INFO:journalctl@ceph.mon.a.smithi046.stdout: checksum: 1 2024-06-14T22:31:56.036 INFO:journalctl@ceph.mon.a.smithi046.stdout: no_block_cache: 0 2024-06-14T22:31:56.036 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache: 0x560c358a11f0 2024-06-14T22:31:56.036 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_name: BinnedLRUCache 2024-06-14T22:31:56.036 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_options: 2024-06-14T22:31:56.036 INFO:journalctl@ceph.mon.a.smithi046.stdout: capacity : 536870912 2024-06-14T22:31:56.036 INFO:journalctl@ceph.mon.a.smithi046.stdout: num_shard_bits : 4 2024-06-14T22:31:56.036 INFO:journalctl@ceph.mon.a.smithi046.stdout: strict_capacity_limit : 0 2024-06-14T22:31:56.037 INFO:journalctl@ceph.mon.a.smithi046.stdout: high_pri_pool_ratio: 0.000 2024-06-14T22:31:56.037 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_compressed: (nil) 2024-06-14T22:31:56.037 INFO:journalctl@ceph.mon.a.smithi046.stdout: persistent_cache: (nil) 2024-06-14T22:31:56.037 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_size: 4096 2024-06-14T22:31:56.037 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_size_deviation: 10 2024-06-14T22:31:56.037 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_restart_interval: 16 2024-06-14T22:31:56.037 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_block_restart_interval: 1 2024-06-14T22:31:56.037 INFO:journalctl@ceph.mon.a.smithi046.stdout: metadata_block_size: 4096 2024-06-14T22:31:56.037 INFO:journalctl@ceph.mon.a.smithi046.stdout: partition_filters: 0 2024-06-14T22:31:56.037 INFO:journalctl@ceph.mon.a.smithi046.stdout: use_delta_encoding: 1 2024-06-14T22:31:56.037 INFO:journalctl@ceph.mon.a.smithi046.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-06-14T22:31:56.037 INFO:journalctl@ceph.mon.a.smithi046.stdout: whole_key_filtering: 1 2024-06-14T22:31:56.037 INFO:journalctl@ceph.mon.a.smithi046.stdout: verify_compression: 0 2024-06-14T22:31:56.038 INFO:journalctl@ceph.mon.a.smithi046.stdout: read_amp_bytes_per_bit: 0 2024-06-14T22:31:56.038 INFO:journalctl@ceph.mon.a.smithi046.stdout: format_version: 4 2024-06-14T22:31:56.038 INFO:journalctl@ceph.mon.a.smithi046.stdout: enable_index_compression: 1 2024-06-14T22:31:56.038 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_align: 0 2024-06-14T22:31:56.038 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.write_buffer_size: 33554432 2024-06-14T22:31:56.038 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_write_buffer_number: 2 2024-06-14T22:31:56.038 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compression: NoCompression 2024-06-14T22:31:56.038 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.bottommost_compression: Disabled 2024-06-14T22:31:56.038 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.prefix_extractor: nullptr 2024-06-14T22:31:56.038 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-14T22:31:56.038 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.num_levels: 7 2024-06-14T22:31:56.038 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-14T22:31:56.038 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-14T22:31:56.039 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-14T22:31:56.039 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-14T22:31:56.039 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-14T22:31:56.039 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-14T22:31:56.039 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-14T22:31:56.039 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-14T22:31:56.039 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-14T22:31:56.039 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-14T22:31:56.039 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-14T22:31:56.039 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compression_opts.level: 32767 2024-06-14T22:31:56.039 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compression_opts.strategy: 0 2024-06-14T22:31:56.039 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-14T22:31:56.039 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-14T22:31:56.040 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-14T22:31:56.040 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compression_opts.enabled: false 2024-06-14T22:31:56.040 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-14T22:31:56.040 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-14T22:31:56.040 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-14T22:31:56.040 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.target_file_size_base: 67108864 2024-06-14T22:31:56.040 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-14T22:31:56.040 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-14T22:31:56.040 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-14T22:31:56.040 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-14T22:31:56.040 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-14T22:31:56.040 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-14T22:31:56.041 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-14T22:31:56.041 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-14T22:31:56.041 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-14T22:31:56.041 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-14T22:31:56.041 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-14T22:31:56.041 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-14T22:31:56.041 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-14T22:31:56.041 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.arena_block_size: 4194304 2024-06-14T22:31:56.041 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-14T22:31:56.041 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-14T22:31:56.041 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-06-14T22:31:56.041 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.disable_auto_compactions: 0 2024-06-14T22:31:56.041 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-14T22:31:56.042 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-14T22:31:56.042 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-14T22:31:56.042 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-14T22:31:56.042 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-14T22:31:56.042 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-14T22:31:56.042 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-14T22:31:56.042 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-14T22:31:56.042 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-14T22:31:56.042 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-14T22:31:56.042 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.table_properties_collectors: 2024-06-14T22:31:56.042 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.inplace_update_support: 0 2024-06-14T22:31:56.042 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-14T22:31:56.043 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-14T22:31:56.043 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-14T22:31:56.043 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-14T22:31:56.043 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.bloom_locality: 0 2024-06-14T22:31:56.043 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.max_successive_merges: 0 2024-06-14T22:31:56.043 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-14T22:31:56.043 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.paranoid_file_checks: 0 2024-06-14T22:31:56.043 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.force_consistency_checks: 1 2024-06-14T22:31:56.043 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.report_bg_io_stats: 0 2024-06-14T22:31:56.043 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.ttl: 2592000 2024-06-14T22:31:56.043 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-14T22:31:56.043 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.enable_blob_files: false 2024-06-14T22:31:56.043 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.min_blob_size: 0 2024-06-14T22:31:56.044 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.blob_file_size: 268435456 2024-06-14T22:31:56.044 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-14T22:31:56.044 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-14T22:31:56.044 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-14T22:31:56.044 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-14T22:31:56.044 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-06-14T22:31:56.044 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-06-14T22:31:56.044 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718404315799754, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-06-14T22:31:56.044 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-06-14T22:31:56.044 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-06-14T22:31:56.044 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718404315800552, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 80261, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 78529, "index_size": 220, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10136, "raw_average_key_size": 47, "raw_value_size": 72736, "raw_average_value_size": 341, "num_data_blocks": 10, "num_entries": 213, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1718404315, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "77ebbc2d-47de-4e31-81aa-3bc05406d264", "db_session_id": "JYE5V9VNQDHAA8KM4SSU"}} 2024-06-14T22:31:56.044 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-06-14T22:31:56.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718404315800910, "job": 1, "event": "recovery_finished"} 2024-06-14T22:31:56.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-14T22:31:56.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x560c358d4700 2024-06-14T22:31:56.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: DB pointer 0x560c358a4000 2024-06-14T22:31:56.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-06-14T22:31:56.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-06-14T22:31:56.045 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** DB Stats ** 2024-06-14T22:31:56.045 INFO:journalctl@ceph.mon.a.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-14T22:31:56.045 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-14T22:31:56.045 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-14T22:31:56.045 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-14T22:31:56.045 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-14T22:31:56.045 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-06-14T22:31:56.046 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-14T22:31:56.046 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-06-14T22:31:56.046 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** Compaction Stats [default] ** 2024-06-14T22:31:56.046 INFO:journalctl@ceph.mon.a.smithi046.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-14T22:31:56.046 INFO:journalctl@ceph.mon.a.smithi046.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-14T22:31:56.046 INFO:journalctl@ceph.mon.a.smithi046.stdout: L0 2/0 80.14 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.1 0.00 0.00 1 0.001 0 0 2024-06-14T22:31:56.046 INFO:journalctl@ceph.mon.a.smithi046.stdout: Sum 2/0 80.14 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.1 0.00 0.00 1 0.001 0 0 2024-06-14T22:31:56.046 INFO:journalctl@ceph.mon.a.smithi046.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.1 0.00 0.00 1 0.001 0 0 2024-06-14T22:31:56.046 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-06-14T22:31:56.046 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** Compaction Stats [default] ** 2024-06-14T22:31:56.046 INFO:journalctl@ceph.mon.a.smithi046.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-14T22:31:56.047 INFO:journalctl@ceph.mon.a.smithi046.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-14T22:31:56.047 INFO:journalctl@ceph.mon.a.smithi046.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 146.1 0.00 0.00 1 0.001 0 0 2024-06-14T22:31:56.047 INFO:journalctl@ceph.mon.a.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-14T22:31:56.047 INFO:journalctl@ceph.mon.a.smithi046.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-14T22:31:56.047 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-14T22:31:56.047 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-14T22:31:56.047 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-14T22:31:56.047 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-14T22:31:56.047 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative compaction: 0.00 GB write, 23.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-14T22:31:56.047 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval compaction: 0.00 GB write, 23.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-14T22:31:56.047 INFO:journalctl@ceph.mon.a.smithi046.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-14T22:31:56.047 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-06-14T22:31:56.047 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-14T22:31:56.048 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-06-14T22:31:56.048 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** Compaction Stats [default] ** 2024-06-14T22:31:56.048 INFO:journalctl@ceph.mon.a.smithi046.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-14T22:31:56.048 INFO:journalctl@ceph.mon.a.smithi046.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-14T22:31:56.048 INFO:journalctl@ceph.mon.a.smithi046.stdout: L0 2/0 80.14 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.1 0.00 0.00 1 0.001 0 0 2024-06-14T22:31:56.048 INFO:journalctl@ceph.mon.a.smithi046.stdout: Sum 2/0 80.14 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 146.1 0.00 0.00 1 0.001 0 0 2024-06-14T22:31:56.048 INFO:journalctl@ceph.mon.a.smithi046.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-06-14T22:31:56.048 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-06-14T22:31:56.048 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** Compaction Stats [default] ** 2024-06-14T22:31:56.048 INFO:journalctl@ceph.mon.a.smithi046.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-14T22:31:56.048 INFO:journalctl@ceph.mon.a.smithi046.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-14T22:31:56.048 INFO:journalctl@ceph.mon.a.smithi046.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 146.1 0.00 0.00 1 0.001 0 0 2024-06-14T22:31:56.048 INFO:journalctl@ceph.mon.a.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-14T22:31:56.049 INFO:journalctl@ceph.mon.a.smithi046.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-14T22:31:56.049 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-14T22:31:56.049 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-14T22:31:56.049 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-14T22:31:56.049 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-14T22:31:56.049 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative compaction: 0.00 GB write, 23.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-14T22:31:56.049 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-14T22:31:56.049 INFO:journalctl@ceph.mon.a.smithi046.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-14T22:31:56.049 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-06-14T22:31:56.049 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-14T22:31:56.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: starting mon.a rank 0 at public addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] at bind addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:31:56.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: mon.a@-1(???) e1 preinit fsid cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:31:56.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: mon.a@-1(???).mds e1 new map 2024-06-14T22:31:56.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: mon.a@-1(???).mds e1 print_map 2024-06-14T22:31:56.050 INFO:journalctl@ceph.mon.a.smithi046.stdout: e1 2024-06-14T22:31:56.050 INFO:journalctl@ceph.mon.a.smithi046.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-14T22:31:56.050 INFO:journalctl@ceph.mon.a.smithi046.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-14T22:31:56.050 INFO:journalctl@ceph.mon.a.smithi046.stdout: legacy client fscid: -1 2024-06-14T22:31:56.050 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-06-14T22:31:56.050 INFO:journalctl@ceph.mon.a.smithi046.stdout: No filesystems configured 2024-06-14T22:31:56.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-14T22:31:56.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-14T22:31:56.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-14T22:31:56.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-14T22:31:56.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-14T22:31:56.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 bash[29548]: 4ab3984515da7ca2360c97fdd9f950c9bb2ad086792cff640dc8fb9b6204b5b3 2024-06-14T22:31:56.051 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 systemd[1]: Started Ceph mon.a for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:31:56.051 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-14T22:31:56.051 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: monmap e1: 1 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-14T22:31:56.051 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: fsmap 2024-06-14T22:31:56.051 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: osdmap e1: 0 total, 0 up, 0 in 2024-06-14T22:31:56.051 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:55 smithi046 ceph-mon[29586]: mgrmap e1: no daemons active 2024-06-14T22:31:57.017 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-14T22:31:57.017 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-14T22:31:57.017 INFO:teuthology.orchestra.run.smithi046.stdout:Creating mgr... 2024-06-14T22:31:57.017 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying port 9283 ... 2024-06-14T22:31:57.198 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mgr.y 2024-06-14T22:31:57.198 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to reset failed state of unit ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mgr.y.service: Unit ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mgr.y.service not loaded. 2024-06-14T22:31:57.352 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-cb009720-2a9d-11ef-bc9d-c7b262605968.target.wants/ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@.service. 2024-06-14T22:31:57.506 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:57 smithi046 systemd[1]: Starting Ceph mgr.y for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:31:57.800 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:57 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/1075812436' entity='client.admin' 2024-06-14T22:31:57.800 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:57 smithi046 podman[29828]: 2024-06-14 22:31:57.594256223 +0000 UTC m=+0.114380090 container create 47a0051cfde6c978a244f73690a092efae8641952d5ea343922c34e842f4bf5a (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240612, RELEASE=quincy-e47ff4b, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-14T22:31:57.800 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:57 smithi046 podman[29828]: 2024-06-14 22:31:57.49743891 +0000 UTC m=+0.017562790 image pull 37f9eedc23c32ee6d04590e8a81233336d2c14cd278f52a2a2746a97e6ef2d95 quay.ceph.io/ceph-ci/ceph:quincy 2024-06-14T22:31:57.800 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:57 smithi046 podman[29828]: 2024-06-14 22:31:57.80548396 +0000 UTC m=+0.325607841 container init 47a0051cfde6c978a244f73690a092efae8641952d5ea343922c34e842f4bf5a (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=quincy-e47ff4b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-06-14T22:31:57.883 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-14T22:31:57.883 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-06-14T22:31:57.892 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-14T22:31:57.892 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-06-14T22:31:57.892 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-06-14T22:31:57.892 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to enable service . firewalld.service is not available 2024-06-14T22:31:57.901 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-14T22:31:57.901 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-06-14T22:31:57.909 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-14T22:31:57.909 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-06-14T22:31:57.909 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-06-14T22:31:57.910 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to open ports <[9283]>. firewalld.service is not available 2024-06-14T22:31:57.910 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr to start... 2024-06-14T22:31:57.910 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr... 2024-06-14T22:31:58.051 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:57 smithi046 podman[29828]: 2024-06-14 22:31:57.811658175 +0000 UTC m=+0.331782042 container start 47a0051cfde6c978a244f73690a092efae8641952d5ea343922c34e842f4bf5a (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y, RELEASE=quincy-e47ff4b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240612, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-14T22:31:58.051 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:57 smithi046 bash[29828]: 47a0051cfde6c978a244f73690a092efae8641952d5ea343922c34e842f4bf5a 2024-06-14T22:31:58.051 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:57 smithi046 systemd[1]: Started Ceph mgr.y for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:31:58.051 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:57 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:31:57.941+0000 7f481d1b5ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-14T22:31:58.302 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:58 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:31:58.075+0000 7f481d1b5ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-14T22:31:58.302 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:58 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:31:58.146+0000 7f481d1b5ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-14T22:31:58.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:31:58 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/500980374' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-14T22:31:58.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:58 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:31:58.416+0000 7f481d1b5ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-14T22:31:58.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:58 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:31:58.479+0000 7f481d1b5ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-14T22:31:59.023 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-06-14T22:31:59.023 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-06-14T22:31:59.023 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "cb009720-2a9d-11ef-bc9d-c7b262605968", 2024-06-14T22:31:59.023 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-06-14T22:31:59.023 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-14T22:31:59.023 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-14T22:31:59.023 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-14T22:31:59.024 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:31:59.024 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-14T22:31:59.024 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-14T22:31:59.024 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-06-14T22:31:59.024 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-06-14T22:31:59.024 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-14T22:31:59.024 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-06-14T22:31:59.024 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-06-14T22:31:59.024 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-06-14T22:31:59.025 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-14T22:31:59.025 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-14T22:31:59.025 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-06-14T22:31:59.025 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-14T22:31:59.025 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:31:59.025 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-14T22:31:59.025 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-14T22:31:59.025 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-14T22:31:59.025 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-14T22:31:59.025 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-14T22:31:59.025 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-14T22:31:59.025 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-14T22:31:59.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-14T22:31:59.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:31:59.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-14T22:31:59.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-14T22:31:59.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-14T22:31:59.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-14T22:31:59.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-14T22:31:59.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-14T22:31:59.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-14T22:31:59.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-14T22:31:59.026 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-14T22:31:59.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:31:59.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-14T22:31:59.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-14T22:31:59.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-14T22:31:59.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-14T22:31:59.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:31:59.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-14T22:31:59.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-14T22:31:59.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-14T22:31:59.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-14T22:31:59.027 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-14T22:31:59.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-14T22:31:59.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-06-14T22:31:59.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-06-14T22:31:59.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-14T22:31:59.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:31:59.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-14T22:31:59.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-14T22:31:59.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-06-14T22:31:50.709809+0000", 2024-06-14T22:31:59.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-14T22:31:59.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:31:59.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-14T22:31:59.028 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-06-14T22:31:59.029 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (1/15)... 2024-06-14T22:31:59.192 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:58 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:31:58.810+0000 7f481d1b5ec0 -1 mgr[py] Module not found: 'k8sevents' 2024-06-14T22:31:59.192 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:58 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:31:58.810+0000 7f481d1b5ec0 -1 mgr[py] Traceback (most recent call last): 2024-06-14T22:31:59.192 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:58 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/usr/share/ceph/mgr/k8sevents/__init__.py", line 1, in 2024-06-14T22:31:59.192 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:58 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: from .module import Module 2024-06-14T22:31:59.192 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:58 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/usr/share/ceph/mgr/k8sevents/module.py", line 70, in 2024-06-14T22:31:59.192 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:58 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: from kubernetes.client.models.v1_event import V1Event 2024-06-14T22:31:59.192 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:58 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ModuleNotFoundError: No module named 'kubernetes.client.models.v1_event' 2024-06-14T22:31:59.192 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:58 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:31:59.192 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:58 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:31:58.811+0000 7f481d1b5ec0 -1 mgr[py] Class not found in module 'k8sevents' 2024-06-14T22:31:59.192 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:58 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:31:58.811+0000 7f481d1b5ec0 -1 mgr[py] Error loading module 'k8sevents': (2) No such file or directory 2024-06-14T22:31:59.830 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:59 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:31:59.456+0000 7f481d1b5ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-14T22:31:59.830 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:59 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:31:59.574+0000 7f481d1b5ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-14T22:31:59.830 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:59 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:31:59.632+0000 7f481d1b5ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-14T22:31:59.830 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:59 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:31:59.834+0000 7f481d1b5ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-14T22:32:00.093 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:59 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:31:59.897+0000 7f481d1b5ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-14T22:32:00.094 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:31:59 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:31:59.966+0000 7f481d1b5ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-14T22:32:00.094 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:00 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:00.097+0000 7f481d1b5ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-14T22:32:00.843 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:00 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:00.537+0000 7f481d1b5ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-14T22:32:01.146 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:00 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:00.847+0000 7f481d1b5ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-14T22:32:01.146 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:00 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:00.916+0000 7f481d1b5ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-14T22:32:01.146 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:01 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:01.054+0000 7f481d1b5ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-14T22:32:01.408 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:01 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:01.348+0000 7f481d1b5ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-14T22:32:01.675 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:01 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/2555093708' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-14T22:32:01.676 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:01 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:01.412+0000 7f481d1b5ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-14T22:32:01.676 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:01 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:01.612+0000 7f481d1b5ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-14T22:32:01.676 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:01 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:01.679+0000 7f481d1b5ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-14T22:32:01.942 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:01 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:01.767+0000 7f481d1b5ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-14T22:32:01.942 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:01 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:01.828+0000 7f481d1b5ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-14T22:32:02.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-06-14T22:32:02.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-06-14T22:32:02.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "cb009720-2a9d-11ef-bc9d-c7b262605968", 2024-06-14T22:32:02.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-06-14T22:32:02.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-14T22:32:02.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-14T22:32:02.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-14T22:32:02.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:32:02.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-14T22:32:02.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-14T22:32:02.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-06-14T22:32:02.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-06-14T22:32:02.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-14T22:32:02.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-06-14T22:32:02.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-06-14T22:32:02.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-06-14T22:32:02.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-14T22:32:02.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-14T22:32:02.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-06-14T22:32:02.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-14T22:32:02.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:32:02.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-14T22:32:02.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-14T22:32:02.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-14T22:32:02.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-14T22:32:02.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-14T22:32:02.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-14T22:32:02.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-14T22:32:02.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-14T22:32:02.149 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:32:02.149 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-14T22:32:02.149 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-14T22:32:02.149 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-14T22:32:02.150 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-14T22:32:02.150 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-14T22:32:02.150 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-14T22:32:02.150 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-14T22:32:02.150 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-14T22:32:02.150 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-14T22:32:02.150 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:32:02.150 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-14T22:32:02.150 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-14T22:32:02.150 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-14T22:32:02.150 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-14T22:32:02.150 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:32:02.151 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-14T22:32:02.151 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-14T22:32:02.151 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-14T22:32:02.151 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-14T22:32:02.151 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-14T22:32:02.151 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-14T22:32:02.151 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-06-14T22:32:02.151 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-06-14T22:32:02.151 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-14T22:32:02.151 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:32:02.151 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-14T22:32:02.151 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-14T22:32:02.152 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-06-14T22:31:50.709809+0000", 2024-06-14T22:32:02.152 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-14T22:32:02.152 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:32:02.152 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-14T22:32:02.152 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-06-14T22:32:02.152 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (2/15)... 2024-06-14T22:32:02.639 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:02 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:02.388+0000 7f481d1b5ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-14T22:32:02.639 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:02 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:02.533+0000 7f481d1b5ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-14T22:32:02.639 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:02 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:02.533+0000 7f481d1b5ec0 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: k8sevents 2024-06-14T22:32:02.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:02 smithi046 ceph-mon[29586]: Activating manager daemon y 2024-06-14T22:32:02.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:02 smithi046 ceph-mon[29586]: mgrmap e2: y(active, starting, since 0.00277195s) 2024-06-14T22:32:02.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:02 smithi046 ceph-mon[29586]: from='mgr.14100 172.21.15.46:0/3572261502' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-14T22:32:02.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:02 smithi046 ceph-mon[29586]: from='mgr.14100 172.21.15.46:0/3572261502' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-14T22:32:02.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:02 smithi046 ceph-mon[29586]: from='mgr.14100 172.21.15.46:0/3572261502' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-14T22:32:02.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:02 smithi046 ceph-mon[29586]: from='mgr.14100 172.21.15.46:0/3572261502' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:32:02.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:02 smithi046 ceph-mon[29586]: from='mgr.14100 172.21.15.46:0/3572261502' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-14T22:32:02.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:02 smithi046 ceph-mon[29586]: Manager daemon y is now available 2024-06-14T22:32:02.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:02 smithi046 ceph-mon[29586]: from='mgr.14100 172.21.15.46:0/3572261502' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-14T22:32:02.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:02 smithi046 ceph-mon[29586]: from='mgr.14100 172.21.15.46:0/3572261502' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-14T22:32:02.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:02 smithi046 ceph-mon[29586]: from='mgr.14100 172.21.15.46:0/3572261502' entity='mgr.y' 2024-06-14T22:32:02.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:02 smithi046 ceph-mon[29586]: from='mgr.14100 172.21.15.46:0/3572261502' entity='mgr.y' 2024-06-14T22:32:02.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:02 smithi046 ceph-mon[29586]: from='mgr.14100 172.21.15.46:0/3572261502' entity='mgr.y' 2024-06-14T22:32:04.827 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:04 smithi046 ceph-mon[29586]: mgrmap e3: y(active, since 1.00713s) 2024-06-14T22:32:05.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:05 smithi046 ceph-mon[29586]: mgrmap e4: y(active, since 2s) 2024-06-14T22:32:05.965 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:05 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/2742536053' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-14T22:32:10.138 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-06-14T22:32:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-06-14T22:32:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "cb009720-2a9d-11ef-bc9d-c7b262605968", 2024-06-14T22:32:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-06-14T22:32:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-14T22:32:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-14T22:32:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-14T22:32:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:32:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-14T22:32:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-14T22:32:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-06-14T22:32:10.139 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-06-14T22:32:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-14T22:32:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-06-14T22:32:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-06-14T22:32:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-06-14T22:32:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-14T22:32:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-14T22:32:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-06-14T22:32:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-14T22:32:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:32:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-14T22:32:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-14T22:32:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-14T22:32:10.140 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-14T22:32:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-14T22:32:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-14T22:32:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-14T22:32:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-14T22:32:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:32:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-14T22:32:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-14T22:32:10.141 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-14T22:32:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-14T22:32:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-14T22:32:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-14T22:32:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-14T22:32:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-14T22:32:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-14T22:32:10.144 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:32:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-14T22:32:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-14T22:32:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-14T22:32:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-14T22:32:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:32:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-14T22:32:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-14T22:32:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-14T22:32:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-14T22:32:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-14T22:32:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-14T22:32:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-06-14T22:32:10.145 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-06-14T22:32:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-14T22:32:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:32:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-14T22:32:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-14T22:32:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-06-14T22:31:50.709809+0000", 2024-06-14T22:32:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-14T22:32:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-06-14T22:32:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-14T22:32:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-06-14T22:32:10.146 INFO:teuthology.orchestra.run.smithi046.stdout:mgr is available 2024-06-14T22:32:13.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:13 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/1102750761' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-14T22:32:16.790 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-06-14T22:32:16.790 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [global] 2024-06-14T22:32:16.790 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout fsid = cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:32:16.790 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-14T22:32:16.790 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-14T22:32:16.790 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-14T22:32:16.791 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-14T22:32:16.791 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-06-14T22:32:16.791 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-14T22:32:16.791 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-14T22:32:16.791 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-06-14T22:32:16.791 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [osd] 2024-06-14T22:32:16.791 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-14T22:32:16.791 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-14T22:32:16.791 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling cephadm module... 2024-06-14T22:32:17.787 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:17 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/2285242782' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-14T22:32:17.787 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:17 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ignoring --setuser ceph since I am not root 2024-06-14T22:32:17.787 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:17 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ignoring --setgroup ceph since I am not root 2024-06-14T22:32:17.788 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:17 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:17.654+0000 7f9d39bdeec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-14T22:32:17.788 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:17 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:17.790+0000 7f9d39bdeec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-14T22:32:18.117 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:17 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:17.863+0000 7f9d39bdeec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-14T22:32:18.117 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:18 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:18.121+0000 7f9d39bdeec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-14T22:32:18.378 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:18 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:18.183+0000 7f9d39bdeec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-14T22:32:18.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:18 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/2285242782' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-14T22:32:18.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:18 smithi046 ceph-mon[29586]: mgrmap e5: y(active, since 14s) 2024-06-14T22:32:18.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:18 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:18.505+0000 7f9d39bdeec0 -1 mgr[py] Module not found: 'k8sevents' 2024-06-14T22:32:18.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:18 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:18.505+0000 7f9d39bdeec0 -1 mgr[py] Traceback (most recent call last): 2024-06-14T22:32:18.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:18 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/usr/share/ceph/mgr/k8sevents/__init__.py", line 1, in 2024-06-14T22:32:18.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:18 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: from .module import Module 2024-06-14T22:32:18.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:18 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/usr/share/ceph/mgr/k8sevents/module.py", line 70, in 2024-06-14T22:32:18.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:18 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: from kubernetes.client.models.v1_event import V1Event 2024-06-14T22:32:18.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:18 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ModuleNotFoundError: No module named 'kubernetes.client.models.v1_event' 2024-06-14T22:32:18.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:18 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:18.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:18 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:18.505+0000 7f9d39bdeec0 -1 mgr[py] Class not found in module 'k8sevents' 2024-06-14T22:32:18.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:18 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:18.505+0000 7f9d39bdeec0 -1 mgr[py] Error loading module 'k8sevents': (2) No such file or directory 2024-06-14T22:32:19.337 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:19 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:19.164+0000 7f9d39bdeec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-14T22:32:19.337 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:19 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:19.282+0000 7f9d39bdeec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-14T22:32:19.337 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:19 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:19.340+0000 7f9d39bdeec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-14T22:32:19.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-06-14T22:32:19.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-14T22:32:19.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-14T22:32:19.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-06-14T22:32:19.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-14T22:32:19.392 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-06-14T22:32:19.392 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for the mgr to restart... 2024-06-14T22:32:19.392 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr epoch 5... 2024-06-14T22:32:19.607 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:19 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/3629051287' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-14T22:32:19.607 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:19 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:19.547+0000 7f9d39bdeec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-14T22:32:19.942 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:19 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:19.611+0000 7f9d39bdeec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-14T22:32:19.942 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:19 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:19.686+0000 7f9d39bdeec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-14T22:32:19.942 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:19 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:19.820+0000 7f9d39bdeec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-14T22:32:20.571 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:20 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:20.263+0000 7f9d39bdeec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-14T22:32:20.942 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:20 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:20.574+0000 7f9d39bdeec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-14T22:32:20.942 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:20 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:20.643+0000 7f9d39bdeec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-14T22:32:20.942 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:20 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:20.779+0000 7f9d39bdeec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-14T22:32:21.301 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:21 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:21.049+0000 7f9d39bdeec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-14T22:32:21.302 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:21 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:21.112+0000 7f9d39bdeec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-14T22:32:21.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:21 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:21.305+0000 7f9d39bdeec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-14T22:32:21.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:21 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:21.371+0000 7f9d39bdeec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-14T22:32:21.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:21 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:21.458+0000 7f9d39bdeec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-14T22:32:21.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:21 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:21.519+0000 7f9d39bdeec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-14T22:32:22.442 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:22.075+0000 7f9d39bdeec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-14T22:32:22.442 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:22.223+0000 7f9d39bdeec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-14T22:32:22.442 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:22.223+0000 7f9d39bdeec0 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: k8sevents 2024-06-14T22:32:22.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-mon[29586]: Active manager daemon y restarted 2024-06-14T22:32:22.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-mon[29586]: Activating manager daemon y 2024-06-14T22:32:22.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-mon[29586]: osdmap e2: 0 total, 0 up, 0 in 2024-06-14T22:32:22.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-mon[29586]: mgrmap e6: y(active, starting, since 0.00299505s) 2024-06-14T22:32:22.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:32:22.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-14T22:32:22.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-14T22:32:22.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-14T22:32:22.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-14T22:32:22.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-mon[29586]: Manager daemon y is now available 2024-06-14T22:32:22.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' 2024-06-14T22:32:22.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' 2024-06-14T22:32:22.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:32:22.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:32:22.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:32:22.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-14T22:32:22.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:22 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-14T22:32:23.527 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:23 smithi046 ceph-mon[29586]: Found migration_current of "None". Setting to last migration. 2024-06-14T22:32:23.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:23 smithi046 ceph-mon[29586]: mgrmap e7: y(active, since 1.00492s) 2024-06-14T22:32:23.789 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-06-14T22:32:23.789 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-14T22:32:23.789 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-14T22:32:23.789 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-06-14T22:32:23.790 INFO:teuthology.orchestra.run.smithi046.stdout:mgr epoch 5 is available 2024-06-14T22:32:23.790 INFO:teuthology.orchestra.run.smithi046.stdout:Setting orchestrator backend to cephadm... 2024-06-14T22:32:24.426 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-mon[29586]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-14T22:32:24.427 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-mon[29586]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-14T22:32:24.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:24.458+0000 7f9cec06a640 -1 log_channel(cephadm) log [ERR] : [14/Jun/2024:22:32:24] ENGINE Error in HTTPServer.serve 2024-06-14T22:32:24.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: Traceback (most recent call last): 2024-06-14T22:32:24.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-14T22:32:24.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: self._connections.run(self.expiration_interval) 2024-06-14T22:32:24.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-14T22:32:24.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: self._run(expiration_interval) 2024-06-14T22:32:24.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-14T22:32:24.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: new_conn = self._from_server_socket(self.server.socket) 2024-06-14T22:32:24.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-14T22:32:24.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-14T22:32:24.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-14T22:32:24.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: s = self.context.wrap_socket( 2024-06-14T22:32:24.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-14T22:32:24.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: return self.sslsocket_class._create( 2024-06-14T22:32:24.694 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-14T22:32:24.694 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: self.do_handshake() 2024-06-14T22:32:24.694 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-14T22:32:24.694 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: self._sslobj.do_handshake() 2024-06-14T22:32:24.694 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-14T22:32:24.694 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:24 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:25.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:25 smithi046 ceph-mon[29586]: mgrmap e8: y(active, since 2s) 2024-06-14T22:32:25.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:25 smithi046 ceph-mon[29586]: [14/Jun/2024:22:32:24] ENGINE Bus STARTING 2024-06-14T22:32:25.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:25 smithi046 ceph-mon[29586]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:32:25.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:25 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' 2024-06-14T22:32:25.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:25 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:32:25.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:25 smithi046 ceph-mon[29586]: [14/Jun/2024:22:32:24] ENGINE Serving on https://172.21.15.46:7150 2024-06-14T22:32:25.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:25 smithi046 ceph-mon[29586]: [14/Jun/2024:22:32:24] ENGINE Bus STARTED 2024-06-14T22:32:25.379 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:25 smithi046 ceph-mon[29586]: [14/Jun/2024:22:32:24] ENGINE Error in HTTPServer.serve 2024-06-14T22:32:25.380 INFO:journalctl@ceph.mon.a.smithi046.stdout: Traceback (most recent call last): 2024-06-14T22:32:25.380 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-14T22:32:25.380 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._connections.run(self.expiration_interval) 2024-06-14T22:32:25.380 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-14T22:32:25.380 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._run(expiration_interval) 2024-06-14T22:32:25.380 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-14T22:32:25.380 INFO:journalctl@ceph.mon.a.smithi046.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-06-14T22:32:25.380 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-14T22:32:25.380 INFO:journalctl@ceph.mon.a.smithi046.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-14T22:32:25.380 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-14T22:32:25.380 INFO:journalctl@ceph.mon.a.smithi046.stdout: s = self.context.wrap_socket( 2024-06-14T22:32:25.380 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-14T22:32:25.380 INFO:journalctl@ceph.mon.a.smithi046.stdout: return self.sslsocket_class._create( 2024-06-14T22:32:25.380 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-14T22:32:25.381 INFO:journalctl@ceph.mon.a.smithi046.stdout: self.do_handshake() 2024-06-14T22:32:25.381 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-14T22:32:25.381 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._sslobj.do_handshake() 2024-06-14T22:32:25.381 INFO:journalctl@ceph.mon.a.smithi046.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-14T22:32:25.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:25 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' 2024-06-14T22:32:25.381 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:25 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:32:26.201 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-14T22:32:26.201 INFO:teuthology.orchestra.run.smithi046.stdout:Generating ssh key... 2024-06-14T22:32:26.909 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:26 smithi046 ceph-mon[29586]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:32:27.192 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: Generating public/private rsa key pair. 2024-06-14T22:32:27.192 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: Your identification has been saved in /tmp/tmporxjy1ap/key 2024-06-14T22:32:27.192 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: Your public key has been saved in /tmp/tmporxjy1ap/key.pub 2024-06-14T22:32:27.192 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: The key fingerprint is: 2024-06-14T22:32:27.192 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: SHA256:Cxwwwo3nnNssc4g/rFR+e8nJIT4OVoVmCzncVT3DwG4 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:32:27.192 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: The key's randomart image is: 2024-06-14T22:32:27.193 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: +---[RSA 3072]----+ 2024-06-14T22:32:27.193 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: | ..oo .oo+ | 2024-06-14T22:32:27.193 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: | ooo= o . = | 2024-06-14T22:32:27.193 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: | +=.* .. o | 2024-06-14T22:32:27.193 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: | +* + E | 2024-06-14T22:32:27.193 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: | ..== S. | 2024-06-14T22:32:27.193 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: | .o=.=... | 2024-06-14T22:32:27.193 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: | .o+=.+.+ | 2024-06-14T22:32:27.193 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: | . .+oo.* | 2024-06-14T22:32:27.193 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: | .. ooo | 2024-06-14T22:32:27.193 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: +----[SHA256]-----+ 2024-06-14T22:32:28.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:27 smithi046 ceph-mon[29586]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:32:28.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:27 smithi046 ceph-mon[29586]: Generating ssh key... 2024-06-14T22:32:28.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:27 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' 2024-06-14T22:32:28.049 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:27 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' 2024-06-14T22:32:28.656 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxnBbWXBS1Pk14DSvPaKWmu3+alryt2DKURicUgsphVeB3M56ujvOsP3U2eid1NyzGv4ijmvi1BZt3z8+jCy+qnChRkbcWTUHN5ZBa6Y4nUBZ2zwxKkBltNToRMGC75kenJtvueucqdvxCGR72i7rvwk4ka4lpZr6jWlVYC+MbOnBpIAyP1i152laKZ/b49z8Z2H7pJlIXKQwJLgMLhQuvxXuuDNO9S5lT2WbQP5m0jEyUzcWS3208mHXwHY8m18x14ODsgrIRD9KrKm/VBv2tsUD/26W7ZEDdjdIv/4oTnoY/21aGOiE3A8VD/Q4dmDO7loaRmLoH9awbmaxZgAr88+kLUQ4OMJrdChCTwKnCcmuIuZ0N/IuSeP7zBHRqN9lU20WyC7RTlxg1gpcwY2X/IOZwfCipZTHFOtpGL07j0o8SwoPxxYTpjEo70WGpUfH+LcdoxGVRcpmypdMPlTLsl0uRP4/b3Sbtf7uworkIZzXLdMX9V5oQqPP9xdeR188= ceph-cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:32:28.657 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-14T22:32:28.657 INFO:teuthology.orchestra.run.smithi046.stdout:Adding key to root@localhost authorized_keys... 2024-06-14T22:32:28.657 INFO:teuthology.orchestra.run.smithi046.stdout:Adding host smithi046... 2024-06-14T22:32:29.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:28 smithi046 ceph-mon[29586]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:32:29.983 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:29 smithi046 ceph-mon[29586]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi046", "addr": "172.21.15.46", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:32:31.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:30 smithi046 ceph-mon[29586]: Deploying cephadm binary to smithi046 2024-06-14T22:32:33.246 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Added host 'smithi046' with addr '172.21.15.46' 2024-06-14T22:32:33.247 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying unmanaged mon service... 2024-06-14T22:32:33.880 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:33 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' 2024-06-14T22:32:33.880 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:33 smithi046 ceph-mon[29586]: Added host smithi046 2024-06-14T22:32:33.880 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:33 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:32:34.472 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-14T22:32:34.472 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying unmanaged mgr service... 2024-06-14T22:32:35.031 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:34 smithi046 ceph-mon[29586]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:32:35.031 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:34 smithi046 ceph-mon[29586]: Saving service mon spec with placement count:5 2024-06-14T22:32:35.031 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:34 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' 2024-06-14T22:32:35.663 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-14T22:32:36.376 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:36 smithi046 ceph-mon[29586]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:32:36.377 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:36 smithi046 ceph-mon[29586]: Saving service mgr spec with placement count:2 2024-06-14T22:32:36.377 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:36 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' 2024-06-14T22:32:37.594 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:37 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/3332204918' entity='client.admin' 2024-06-14T22:32:38.636 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:38 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' 2024-06-14T22:32:38.637 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:38 smithi046 ceph-mon[29586]: from='mgr.14118 172.21.15.46:0/2857879698' entity='mgr.y' 2024-06-14T22:32:38.637 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:38 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/2561151526' entity='client.admin' 2024-06-14T22:32:38.639 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling the dashboard module... 2024-06-14T22:32:39.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:39 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/1386895693' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-14T22:32:39.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:39 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ignoring --setuser ceph since I am not root 2024-06-14T22:32:39.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:39 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ignoring --setgroup ceph since I am not root 2024-06-14T22:32:40.153 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:39 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:39.759+0000 7f3f31fe6ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-14T22:32:40.153 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:39 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:39.891+0000 7f3f31fe6ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-14T22:32:40.153 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:39 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:39.961+0000 7f3f31fe6ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-14T22:32:40.426 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:40 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:40.222+0000 7f3f31fe6ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-14T22:32:40.426 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:40 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:40.283+0000 7f3f31fe6ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-14T22:32:40.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:40 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/1386895693' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-14T22:32:40.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:40 smithi046 ceph-mon[29586]: mgrmap e9: y(active, since 17s) 2024-06-14T22:32:40.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:40 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:40.612+0000 7f3f31fe6ec0 -1 mgr[py] Module not found: 'k8sevents' 2024-06-14T22:32:40.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:40 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:40.612+0000 7f3f31fe6ec0 -1 mgr[py] Traceback (most recent call last): 2024-06-14T22:32:40.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:40 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/usr/share/ceph/mgr/k8sevents/__init__.py", line 1, in 2024-06-14T22:32:40.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:40 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: from .module import Module 2024-06-14T22:32:40.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:40 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/usr/share/ceph/mgr/k8sevents/module.py", line 70, in 2024-06-14T22:32:40.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:40 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: from kubernetes.client.models.v1_event import V1Event 2024-06-14T22:32:40.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:40 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ModuleNotFoundError: No module named 'kubernetes.client.models.v1_event' 2024-06-14T22:32:40.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:40 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:40.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:40 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:40.612+0000 7f3f31fe6ec0 -1 mgr[py] Class not found in module 'k8sevents' 2024-06-14T22:32:40.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:40 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:40.612+0000 7f3f31fe6ec0 -1 mgr[py] Error loading module 'k8sevents': (2) No such file or directory 2024-06-14T22:32:41.410 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:41 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:41.268+0000 7f3f31fe6ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-14T22:32:41.410 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:41 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:41.387+0000 7f3f31fe6ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-14T22:32:41.413 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-06-14T22:32:41.413 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-14T22:32:41.413 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-14T22:32:41.413 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-06-14T22:32:41.413 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-14T22:32:41.413 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-06-14T22:32:41.413 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for the mgr to restart... 2024-06-14T22:32:41.413 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr epoch 9... 2024-06-14T22:32:41.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:41 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/3616827211' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-14T22:32:41.660 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:41 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:41.448+0000 7f3f31fe6ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-14T22:32:41.660 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:41 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:41.656+0000 7f3f31fe6ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-14T22:32:41.928 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:41 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:41.722+0000 7f3f31fe6ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-14T22:32:41.928 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:41 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:41.793+0000 7f3f31fe6ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-14T22:32:41.928 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:41 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:41.931+0000 7f3f31fe6ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-14T22:32:42.689 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:42 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:42.380+0000 7f3f31fe6ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-14T22:32:42.689 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:42 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:42.692+0000 7f3f31fe6ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-14T22:32:42.942 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:42 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:42.762+0000 7f3f31fe6ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-14T22:32:42.942 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:42 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:42.898+0000 7f3f31fe6ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-14T22:32:43.421 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:43 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:43.169+0000 7f3f31fe6ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-14T22:32:43.421 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:43 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:43.232+0000 7f3f31fe6ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-14T22:32:43.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:43 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:43.425+0000 7f3f31fe6ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-14T22:32:43.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:43 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:43.491+0000 7f3f31fe6ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-14T22:32:43.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:43 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:43.577+0000 7f3f31fe6ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-14T22:32:43.692 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:43 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:43.638+0000 7f3f31fe6ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-14T22:32:44.442 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:44 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:44.194+0000 7f3f31fe6ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-14T22:32:44.442 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:44 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:44.339+0000 7f3f31fe6ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-14T22:32:44.442 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:44 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:44.339+0000 7f3f31fe6ec0 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: k8sevents 2024-06-14T22:32:44.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:44 smithi046 ceph-mon[29586]: Active manager daemon y restarted 2024-06-14T22:32:44.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:44 smithi046 ceph-mon[29586]: Activating manager daemon y 2024-06-14T22:32:44.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:44 smithi046 ceph-mon[29586]: osdmap e3: 0 total, 0 up, 0 in 2024-06-14T22:32:44.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:44 smithi046 ceph-mon[29586]: mgrmap e10: y(active, starting, since 0.00300869s) 2024-06-14T22:32:44.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:44 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:32:44.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:44 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-14T22:32:44.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:44 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-14T22:32:44.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:44 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-14T22:32:44.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:44 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-14T22:32:44.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:44 smithi046 ceph-mon[29586]: Manager daemon y is now available 2024-06-14T22:32:44.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:44 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:32:44.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:44 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:32:44.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:44 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-14T22:32:44.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:44 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-14T22:32:45.855 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-06-14T22:32:45.855 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-14T22:32:45.855 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-14T22:32:45.855 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-06-14T22:32:45.855 INFO:teuthology.orchestra.run.smithi046.stdout:mgr epoch 9 is available 2024-06-14T22:32:45.855 INFO:teuthology.orchestra.run.smithi046.stdout:Generating a dashboard self-signed certificate... 2024-06-14T22:32:46.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-mon[29586]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-14T22:32:46.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-mon[29586]: mgrmap e11: y(active, since 1.00571s) 2024-06-14T22:32:46.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-mon[29586]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-14T22:32:46.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:32:46.442 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:46.125+0000 7f3ee2c21640 -1 log_channel(cephadm) log [ERR] : [14/Jun/2024:22:32:46] ENGINE Error in HTTPServer.serve 2024-06-14T22:32:46.442 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: Traceback (most recent call last): 2024-06-14T22:32:46.443 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-14T22:32:46.443 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: self._connections.run(self.expiration_interval) 2024-06-14T22:32:46.443 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-14T22:32:46.443 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: self._run(expiration_interval) 2024-06-14T22:32:46.443 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-14T22:32:46.443 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: new_conn = self._from_server_socket(self.server.socket) 2024-06-14T22:32:46.443 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-14T22:32:46.443 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-14T22:32:46.443 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-14T22:32:46.443 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: s = self.context.wrap_socket( 2024-06-14T22:32:46.443 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-14T22:32:46.443 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: return self.sslsocket_class._create( 2024-06-14T22:32:46.444 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-14T22:32:46.444 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: self.do_handshake() 2024-06-14T22:32:46.444 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-14T22:32:46.444 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: self._sslobj.do_handshake() 2024-06-14T22:32:46.444 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-14T22:32:46.444 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:32:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:32:47.544 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:47 smithi046 ceph-mon[29586]: [14/Jun/2024:22:32:46] ENGINE Bus STARTING 2024-06-14T22:32:47.544 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:47 smithi046 ceph-mon[29586]: [14/Jun/2024:22:32:46] ENGINE Serving on https://172.21.15.46:7150 2024-06-14T22:32:47.544 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:47 smithi046 ceph-mon[29586]: [14/Jun/2024:22:32:46] ENGINE Bus STARTED 2024-06-14T22:32:47.544 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:47 smithi046 ceph-mon[29586]: [14/Jun/2024:22:32:46] ENGINE Error in HTTPServer.serve 2024-06-14T22:32:47.544 INFO:journalctl@ceph.mon.a.smithi046.stdout: Traceback (most recent call last): 2024-06-14T22:32:47.544 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-14T22:32:47.544 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._connections.run(self.expiration_interval) 2024-06-14T22:32:47.544 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-14T22:32:47.544 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._run(expiration_interval) 2024-06-14T22:32:47.544 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-14T22:32:47.544 INFO:journalctl@ceph.mon.a.smithi046.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-06-14T22:32:47.545 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-14T22:32:47.545 INFO:journalctl@ceph.mon.a.smithi046.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-14T22:32:47.545 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-14T22:32:47.545 INFO:journalctl@ceph.mon.a.smithi046.stdout: s = self.context.wrap_socket( 2024-06-14T22:32:47.545 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-14T22:32:47.545 INFO:journalctl@ceph.mon.a.smithi046.stdout: return self.sslsocket_class._create( 2024-06-14T22:32:47.545 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-14T22:32:47.545 INFO:journalctl@ceph.mon.a.smithi046.stdout: self.do_handshake() 2024-06-14T22:32:47.545 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-14T22:32:47.545 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._sslobj.do_handshake() 2024-06-14T22:32:47.545 INFO:journalctl@ceph.mon.a.smithi046.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-14T22:32:47.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:47 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:32:47.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:47 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:32:47.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:47 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:32:47.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:47 smithi046 ceph-mon[29586]: mgrmap e12: y(active, since 2s) 2024-06-14T22:32:47.546 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:47 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:32:47.546 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-14T22:32:47.546 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial admin user... 2024-06-14T22:32:48.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:48 smithi046 ceph-mon[29586]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:32:48.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:48 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:32:49.122 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$CKwPcFveSi9fj8gkaWxEMeIyrKfIlAoLImpnM5FWagrogKxDyhw5q", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1718404368, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-14T22:32:49.123 INFO:teuthology.orchestra.run.smithi046.stdout:Fetching dashboard port number... 2024-06-14T22:32:49.564 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:49 smithi046 ceph-mon[29586]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:32:50.562 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:50 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/36038705' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-14T22:32:50.565 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 8443 2024-06-14T22:32:50.575 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-14T22:32:50.576 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-06-14T22:32:50.584 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-14T22:32:50.584 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-06-14T22:32:50.584 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-06-14T22:32:50.584 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-14T22:32:50.586 INFO:teuthology.orchestra.run.smithi046.stdout:Ceph Dashboard is now available at: 2024-06-14T22:32:50.586 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:32:50.586 INFO:teuthology.orchestra.run.smithi046.stdout: URL: https://smithi046.front.sepia.ceph.com:8443/ 2024-06-14T22:32:50.586 INFO:teuthology.orchestra.run.smithi046.stdout: User: admin 2024-06-14T22:32:50.586 INFO:teuthology.orchestra.run.smithi046.stdout: Password: oaeteh2kde 2024-06-14T22:32:50.587 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:32:50.587 INFO:teuthology.orchestra.run.smithi046.stdout:Saving cluster configuration to /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/config directory 2024-06-14T22:32:50.587 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling autotune for osd_memory_target 2024-06-14T22:32:51.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:51 smithi046 ceph-mon[29586]: mgrmap e13: y(active, since 6s) 2024-06-14T22:32:52.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-14T22:32:52.918 INFO:teuthology.orchestra.run.smithi046.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-14T22:32:52.919 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:32:52.919 INFO:teuthology.orchestra.run.smithi046.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-14T22:32:52.919 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:32:52.919 INFO:teuthology.orchestra.run.smithi046.stdout:Or, if you are only running a single cluster on this host: 2024-06-14T22:32:52.919 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:32:52.919 INFO:teuthology.orchestra.run.smithi046.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-14T22:32:52.919 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:32:52.919 INFO:teuthology.orchestra.run.smithi046.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-14T22:32:52.919 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:32:52.919 INFO:teuthology.orchestra.run.smithi046.stdout: ceph telemetry on 2024-06-14T22:32:52.919 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:32:52.919 INFO:teuthology.orchestra.run.smithi046.stdout:For more information see: 2024-06-14T22:32:52.920 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:32:52.920 INFO:teuthology.orchestra.run.smithi046.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-06-14T22:32:52.920 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:32:52.920 INFO:teuthology.orchestra.run.smithi046.stdout:Bootstrap complete. 2024-06-14T22:32:52.963 INFO:tasks.cephadm:Fetching config... 2024-06-14T22:32:52.964 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-06-14T22:32:52.964 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-14T22:32:52.981 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-14T22:32:52.981 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-06-14T22:32:52.982 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-14T22:32:53.038 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-14T22:32:53.038 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-06-14T22:32:53.038 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-14T22:32:53.103 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-14T22:32:53.103 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-06-14T22:32:53.103 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-14T22:32:53.160 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-14T22:32:53.160 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxnBbWXBS1Pk14DSvPaKWmu3+alryt2DKURicUgsphVeB3M56ujvOsP3U2eid1NyzGv4ijmvi1BZt3z8+jCy+qnChRkbcWTUHN5ZBa6Y4nUBZ2zwxKkBltNToRMGC75kenJtvueucqdvxCGR72i7rvwk4ka4lpZr6jWlVYC+MbOnBpIAyP1i152laKZ/b49z8Z2H7pJlIXKQwJLgMLhQuvxXuuDNO9S5lT2WbQP5m0jEyUzcWS3208mHXwHY8m18x14ODsgrIRD9KrKm/VBv2tsUD/26W7ZEDdjdIv/4oTnoY/21aGOiE3A8VD/Q4dmDO7loaRmLoH9awbmaxZgAr88+kLUQ4OMJrdChCTwKnCcmuIuZ0N/IuSeP7zBHRqN9lU20WyC7RTlxg1gpcwY2X/IOZwfCipZTHFOtpGL07j0o8SwoPxxYTpjEo70WGpUfH+LcdoxGVRcpmypdMPlTLsl0uRP4/b3Sbtf7uworkIZzXLdMX9V5oQqPP9xdeR188= ceph-cb009720-2a9d-11ef-bc9d-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-14T22:32:53.251 INFO:teuthology.orchestra.run.smithi046.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxnBbWXBS1Pk14DSvPaKWmu3+alryt2DKURicUgsphVeB3M56ujvOsP3U2eid1NyzGv4ijmvi1BZt3z8+jCy+qnChRkbcWTUHN5ZBa6Y4nUBZ2zwxKkBltNToRMGC75kenJtvueucqdvxCGR72i7rvwk4ka4lpZr6jWlVYC+MbOnBpIAyP1i152laKZ/b49z8Z2H7pJlIXKQwJLgMLhQuvxXuuDNO9S5lT2WbQP5m0jEyUzcWS3208mHXwHY8m18x14ODsgrIRD9KrKm/VBv2tsUD/26W7ZEDdjdIv/4oTnoY/21aGOiE3A8VD/Q4dmDO7loaRmLoH9awbmaxZgAr88+kLUQ4OMJrdChCTwKnCcmuIuZ0N/IuSeP7zBHRqN9lU20WyC7RTlxg1gpcwY2X/IOZwfCipZTHFOtpGL07j0o8SwoPxxYTpjEo70WGpUfH+LcdoxGVRcpmypdMPlTLsl0uRP4/b3Sbtf7uworkIZzXLdMX9V5oQqPP9xdeR188= ceph-cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:32:53.283 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxnBbWXBS1Pk14DSvPaKWmu3+alryt2DKURicUgsphVeB3M56ujvOsP3U2eid1NyzGv4ijmvi1BZt3z8+jCy+qnChRkbcWTUHN5ZBa6Y4nUBZ2zwxKkBltNToRMGC75kenJtvueucqdvxCGR72i7rvwk4ka4lpZr6jWlVYC+MbOnBpIAyP1i152laKZ/b49z8Z2H7pJlIXKQwJLgMLhQuvxXuuDNO9S5lT2WbQP5m0jEyUzcWS3208mHXwHY8m18x14ODsgrIRD9KrKm/VBv2tsUD/26W7ZEDdjdIv/4oTnoY/21aGOiE3A8VD/Q4dmDO7loaRmLoH9awbmaxZgAr88+kLUQ4OMJrdChCTwKnCcmuIuZ0N/IuSeP7zBHRqN9lU20WyC7RTlxg1gpcwY2X/IOZwfCipZTHFOtpGL07j0o8SwoPxxYTpjEo70WGpUfH+LcdoxGVRcpmypdMPlTLsl0uRP4/b3Sbtf7uworkIZzXLdMX9V5oQqPP9xdeR188= ceph-cb009720-2a9d-11ef-bc9d-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-14T22:32:53.324 INFO:teuthology.orchestra.run.smithi178.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCxnBbWXBS1Pk14DSvPaKWmu3+alryt2DKURicUgsphVeB3M56ujvOsP3U2eid1NyzGv4ijmvi1BZt3z8+jCy+qnChRkbcWTUHN5ZBa6Y4nUBZ2zwxKkBltNToRMGC75kenJtvueucqdvxCGR72i7rvwk4ka4lpZr6jWlVYC+MbOnBpIAyP1i152laKZ/b49z8Z2H7pJlIXKQwJLgMLhQuvxXuuDNO9S5lT2WbQP5m0jEyUzcWS3208mHXwHY8m18x14ODsgrIRD9KrKm/VBv2tsUD/26W7ZEDdjdIv/4oTnoY/21aGOiE3A8VD/Q4dmDO7loaRmLoH9awbmaxZgAr88+kLUQ4OMJrdChCTwKnCcmuIuZ0N/IuSeP7zBHRqN9lU20WyC7RTlxg1gpcwY2X/IOZwfCipZTHFOtpGL07j0o8SwoPxxYTpjEo70WGpUfH+LcdoxGVRcpmypdMPlTLsl0uRP4/b3Sbtf7uworkIZzXLdMX9V5oQqPP9xdeR188= ceph-cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:32:53.337 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-14T22:32:53.599 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:32:53.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:53 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/3725980390' entity='client.admin' 2024-06-14T22:32:56.285 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:56 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:32:56.285 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:56 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:32:56.285 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:56 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi046", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:32:56.286 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:56 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:32:56.286 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:56 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:32:56.286 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:56 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:32:56.286 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:56 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/966583665' entity='client.admin' 2024-06-14T22:32:56.286 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:56 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:32:56.286 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:56 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:32:56.310 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-14T22:32:56.311 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-14T22:32:56.568 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:32:59.035 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi178 2024-06-14T22:32:59.035 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-14T22:32:59.035 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.conf 2024-06-14T22:32:59.054 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-14T22:32:59.054 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-14T22:32:59.112 INFO:tasks.cephadm:Adding host smithi178 to orchestrator... 2024-06-14T22:32:59.112 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph orch host add smithi178 2024-06-14T22:32:59.379 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:32:59.574 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:59 smithi046 ceph-mon[29586]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:32:59.574 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:59 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:32:59.574 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:59 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:32:59.574 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:59 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:32:59.575 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:59 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:32:59.575 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:59 smithi046 ceph-mon[29586]: Updating smithi046:/etc/ceph/ceph.conf 2024-06-14T22:32:59.575 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:59 smithi046 ceph-mon[29586]: Updating smithi046:/var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-14T22:32:59.575 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:59 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:32:59.575 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:59 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:32:59.575 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:32:59 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:00.431 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:00 smithi046 ceph-mon[29586]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-06-14T22:33:00.431 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:00 smithi046 ceph-mon[29586]: Updating smithi046:/var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-14T22:33:02.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:01 smithi046 ceph-mon[29586]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:33:03.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:02 smithi046 ceph-mon[29586]: Deploying cephadm binary to smithi178 2024-06-14T22:33:04.475 INFO:teuthology.orchestra.run.smithi046.stdout:Added host 'smithi178' with addr '172.21.15.178' 2024-06-14T22:33:05.055 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph orch host ls --format=json 2024-06-14T22:33:05.313 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:33:05.689 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:05 smithi046 ceph-mon[29586]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:05.689 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:05 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:05.689 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:05 smithi046 ceph-mon[29586]: Added host smithi178 2024-06-14T22:33:05.689 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:05 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:33:05.689 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:05 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:05.690 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:05 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:07.017 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:33:07.017 INFO:teuthology.orchestra.run.smithi046.stdout:[{"addr": "172.21.15.46", "hostname": "smithi046", "labels": [], "status": ""}, {"addr": "172.21.15.178", "hostname": "smithi178", "labels": [], "status": ""}] 2024-06-14T22:33:07.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:06 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:07.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:06 smithi046 ceph-mon[29586]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:07.575 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-14T22:33:07.575 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd crush tunables default 2024-06-14T22:33:07.833 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:33:08.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:07 smithi046 ceph-mon[29586]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-14T22:33:08.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:08 smithi046 ceph-mon[29586]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:08.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:08 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:08.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:08 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:08.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:08 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:08.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:08 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:08.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:08 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:33:08.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:08 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:08.906 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:08 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:33:10.605 INFO:teuthology.orchestra.run.smithi046.stderr:adjusted tunables profile to default 2024-06-14T22:33:10.937 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:10 smithi046 ceph-mon[29586]: Updating smithi178:/etc/ceph/ceph.conf 2024-06-14T22:33:10.937 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:10 smithi046 ceph-mon[29586]: Updating smithi178:/var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-14T22:33:10.937 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:10 smithi046 ceph-mon[29586]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-06-14T22:33:10.937 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:10 smithi046 ceph-mon[29586]: Updating smithi178:/var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-14T22:33:10.937 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:10 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:10.937 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:10 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:10.937 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:10 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:10.937 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:10 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/1314704589' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-14T22:33:11.218 INFO:tasks.cephadm:Adding mon.a on smithi046 2024-06-14T22:33:11.218 INFO:tasks.cephadm:Adding mon.c on smithi046 2024-06-14T22:33:11.218 INFO:tasks.cephadm:Adding mon.b on smithi178 2024-06-14T22:33:11.218 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph orch apply mon '3;smithi046:172.21.15.46=a;smithi046:[v2:172.21.15.46:3301,v1:172.21.15.46:6790]=c;smithi178:172.21.15.178=b' 2024-06-14T22:33:11.467 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-14T22:33:11.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:11 smithi046 ceph-mon[29586]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:11.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:11 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/1314704589' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-14T22:33:11.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:11 smithi046 ceph-mon[29586]: osdmap e4: 0 total, 0 up, 0 in 2024-06-14T22:33:13.302 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mon update... 2024-06-14T22:33:13.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:13 smithi046 ceph-mon[29586]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:13.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:13 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:13.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:13 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:33:13.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:13 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:13.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:13 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:33:13.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:13 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:13.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:13 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-14T22:33:13.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:13 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:14.049 DEBUG:teuthology.orchestra.run.smithi046:mon.c> sudo journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mon.c.service 2024-06-14T22:33:14.052 DEBUG:teuthology.orchestra.run.smithi178:mon.b> sudo journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mon.b.service 2024-06-14T22:33:14.054 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-14T22:33:14.054 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-14T22:33:14.812 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-14T22:33:14.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:14 smithi046 ceph-mon[29586]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi046:[v2:172.21.15.46:3301,v1:172.21.15.46:6790]=c;smithi178:172.21.15.178=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:33:14.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:14 smithi046 ceph-mon[29586]: Saving service mon spec with placement smithi046:172.21.15.46=a;smithi046:[v2:172.21.15.46:3301,v1:172.21.15.46:6790]=c;smithi178:172.21.15.178=b;count:3 2024-06-14T22:33:14.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:14 smithi046 ceph-mon[29586]: Deploying daemon mon.b on smithi178 2024-06-14T22:33:15.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:15 smithi046 ceph-mon[29586]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:17.130 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:33:17.130 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"cb009720-2a9d-11ef-bc9d-c7b262605968","modified":"2024-06-14T22:31:47.571363Z","created":"2024-06-14T22:31:47.571363Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-14T22:33:17.130 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-06-14T22:33:17.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:17 smithi046 ceph-mon[29586]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:17.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:17 smithi046 ceph-mon[29586]: from='client.? 172.21.15.178:0/1490751780' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-14T22:33:18.711 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-14T22:33:18.711 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-14T22:33:19.398 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 podman[32880]: 2024-06-14 22:33:19.384319392 +0000 UTC m=+0.359891063 container start cf042b827361861b30df3fec20ea522876b01fd7706b3f7846734dd486352577 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-b, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=quincy-e47ff4b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240612) 2024-06-14T22:33:19.398 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: set uid:gid to 167:167 (ceph:ceph) 2024-06-14T22:33:19.398 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: ceph version 17.2.7-1008-ge47ff4b8 (e47ff4b87f747658fc7376cc7065fc56801eb37b) quincy (stable), process ceph-mon, pid 7 2024-06-14T22:33:19.399 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: pidfile_write: ignore empty --pid-file 2024-06-14T22:33:19.495 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.b/config 2024-06-14T22:33:19.725 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: load: jerasure load: lrc 2024-06-14T22:33:19.725 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: RocksDB version: 6.15.5 2024-06-14T22:33:19.725 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-06-14T22:33:19.725 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Compile date Jun 14 2024 2024-06-14T22:33:19.725 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: DB SUMMARY 2024-06-14T22:33:19.725 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: DB Session ID: AJZIBPCTN2RGBFTBIDFJ 2024-06-14T22:33:19.725 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: CURRENT file: CURRENT 2024-06-14T22:33:19.725 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: IDENTITY file: IDENTITY 2024-06-14T22:33:19.726 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-06-14T22:33:19.726 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-06-14T22:33:19.726 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-06-14T22:33:19.726 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.error_if_exists: 0 2024-06-14T22:33:19.726 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.create_if_missing: 0 2024-06-14T22:33:19.726 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.paranoid_checks: 1 2024-06-14T22:33:19.726 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-14T22:33:19.726 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.env: 0x5619036c4440 2024-06-14T22:33:19.727 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.fs: Posix File System 2024-06-14T22:33:19.727 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.info_log: 0x561905560f40 2024-06-14T22:33:19.727 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_file_opening_threads: 16 2024-06-14T22:33:19.727 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.statistics: (nil) 2024-06-14T22:33:19.727 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.use_fsync: 0 2024-06-14T22:33:19.727 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_log_file_size: 0 2024-06-14T22:33:19.727 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-14T22:33:19.727 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-14T22:33:19.727 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.keep_log_file_num: 1000 2024-06-14T22:33:19.727 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.recycle_log_file_num: 0 2024-06-14T22:33:19.727 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.allow_fallocate: 1 2024-06-14T22:33:19.727 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.allow_mmap_reads: 0 2024-06-14T22:33:19.727 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.allow_mmap_writes: 0 2024-06-14T22:33:19.728 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.use_direct_reads: 0 2024-06-14T22:33:19.728 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-14T22:33:19.728 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.create_missing_column_families: 0 2024-06-14T22:33:19.728 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.db_log_dir: 2024-06-14T22:33:19.728 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2024-06-14T22:33:19.728 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-14T22:33:19.728 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-14T22:33:19.728 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-14T22:33:19.728 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-14T22:33:19.728 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-14T22:33:19.728 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-14T22:33:19.729 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.advise_random_on_open: 1 2024-06-14T22:33:19.729 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.db_write_buffer_size: 0 2024-06-14T22:33:19.729 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.write_buffer_manager: 0x56190561e7b0 2024-06-14T22:33:19.729 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-14T22:33:19.729 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-06-14T22:33:19.729 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-14T22:33:19.729 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-14T22:33:19.729 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.rate_limiter: (nil) 2024-06-14T22:33:19.729 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-14T22:33:19.729 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.wal_recovery_mode: 2 2024-06-14T22:33:19.729 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.enable_thread_tracking: 0 2024-06-14T22:33:19.729 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.enable_pipelined_write: 0 2024-06-14T22:33:19.729 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.unordered_write: 0 2024-06-14T22:33:19.730 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-14T22:33:19.730 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-14T22:33:19.730 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-14T22:33:19.730 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-14T22:33:19.730 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.row_cache: None 2024-06-14T22:33:19.730 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.wal_filter: None 2024-06-14T22:33:19.730 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-14T22:33:19.730 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.allow_ingest_behind: 0 2024-06-14T22:33:19.730 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.preserve_deletes: 0 2024-06-14T22:33:19.730 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.two_write_queues: 0 2024-06-14T22:33:19.730 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.manual_wal_flush: 0 2024-06-14T22:33:19.730 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.atomic_flush: 0 2024-06-14T22:33:19.731 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-14T22:33:19.731 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-14T22:33:19.731 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-14T22:33:19.731 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.log_readahead_size: 0 2024-06-14T22:33:19.731 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-14T22:33:19.731 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.best_efforts_recovery: 0 2024-06-14T22:33:19.731 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-14T22:33:19.731 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-14T22:33:19.731 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.allow_data_in_errors: 0 2024-06-14T22:33:19.731 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.db_host_id: __hostname__ 2024-06-14T22:33:19.731 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_background_jobs: 2 2024-06-14T22:33:19.731 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_background_compactions: -1 2024-06-14T22:33:19.731 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_subcompactions: 1 2024-06-14T22:33:19.732 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-14T22:33:19.733 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-14T22:33:19.733 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-14T22:33:19.733 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_total_wal_size: 0 2024-06-14T22:33:19.733 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-14T22:33:19.733 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-14T22:33:19.733 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-14T22:33:19.733 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-14T22:33:19.733 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_open_files: -1 2024-06-14T22:33:19.733 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.bytes_per_sync: 0 2024-06-14T22:33:19.733 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-14T22:33:19.734 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-14T22:33:19.734 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compaction_readahead_size: 0 2024-06-14T22:33:19.734 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_background_flushes: -1 2024-06-14T22:33:19.734 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Compression algorithms supported: 2024-06-14T22:33:19.734 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: kZSTD supported: 0 2024-06-14T22:33:19.734 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: kXpressCompression supported: 0 2024-06-14T22:33:19.734 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: kBZip2Compression supported: 0 2024-06-14T22:33:19.734 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-14T22:33:19.734 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: kLZ4Compression supported: 1 2024-06-14T22:33:19.734 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: kZlibCompression supported: 1 2024-06-14T22:33:19.734 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: kLZ4HCCompression supported: 1 2024-06-14T22:33:19.734 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: kSnappyCompression supported: 1 2024-06-14T22:33:19.735 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-14T22:33:19.735 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 2024-06-14T22:33:19.735 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-06-14T22:33:19.735 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-14T22:33:19.735 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.merge_operator: 2024-06-14T22:33:19.735 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compaction_filter: None 2024-06-14T22:33:19.735 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compaction_filter_factory: None 2024-06-14T22:33:19.735 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.sst_partitioner_factory: None 2024-06-14T22:33:19.735 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-14T22:33:19.735 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-14T22:33:19.735 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561905507a68) 2024-06-14T22:33:19.735 INFO:journalctl@ceph.mon.b.smithi178.stdout: cache_index_and_filter_blocks: 1 2024-06-14T22:33:19.735 INFO:journalctl@ceph.mon.b.smithi178.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-14T22:33:19.735 INFO:journalctl@ceph.mon.b.smithi178.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-14T22:33:19.736 INFO:journalctl@ceph.mon.b.smithi178.stdout: pin_top_level_index_and_filter: 1 2024-06-14T22:33:19.736 INFO:journalctl@ceph.mon.b.smithi178.stdout: index_type: 0 2024-06-14T22:33:19.736 INFO:journalctl@ceph.mon.b.smithi178.stdout: data_block_index_type: 0 2024-06-14T22:33:19.736 INFO:journalctl@ceph.mon.b.smithi178.stdout: index_shortening: 1 2024-06-14T22:33:19.736 INFO:journalctl@ceph.mon.b.smithi178.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-14T22:33:19.736 INFO:journalctl@ceph.mon.b.smithi178.stdout: hash_index_allow_collision: 1 2024-06-14T22:33:19.736 INFO:journalctl@ceph.mon.b.smithi178.stdout: checksum: 1 2024-06-14T22:33:19.736 INFO:journalctl@ceph.mon.b.smithi178.stdout: no_block_cache: 0 2024-06-14T22:33:19.736 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_cache: 0x56190553d350 2024-06-14T22:33:19.736 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_cache_name: BinnedLRUCache 2024-06-14T22:33:19.736 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_cache_options: 2024-06-14T22:33:19.736 INFO:journalctl@ceph.mon.b.smithi178.stdout: capacity : 536870912 2024-06-14T22:33:19.737 INFO:journalctl@ceph.mon.b.smithi178.stdout: num_shard_bits : 4 2024-06-14T22:33:19.737 INFO:journalctl@ceph.mon.b.smithi178.stdout: strict_capacity_limit : 0 2024-06-14T22:33:19.737 INFO:journalctl@ceph.mon.b.smithi178.stdout: high_pri_pool_ratio: 0.000 2024-06-14T22:33:19.737 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_cache_compressed: (nil) 2024-06-14T22:33:19.737 INFO:journalctl@ceph.mon.b.smithi178.stdout: persistent_cache: (nil) 2024-06-14T22:33:19.737 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_size: 4096 2024-06-14T22:33:19.737 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_size_deviation: 10 2024-06-14T22:33:19.737 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_restart_interval: 16 2024-06-14T22:33:19.737 INFO:journalctl@ceph.mon.b.smithi178.stdout: index_block_restart_interval: 1 2024-06-14T22:33:19.737 INFO:journalctl@ceph.mon.b.smithi178.stdout: metadata_block_size: 4096 2024-06-14T22:33:19.737 INFO:journalctl@ceph.mon.b.smithi178.stdout: partition_filters: 0 2024-06-14T22:33:19.737 INFO:journalctl@ceph.mon.b.smithi178.stdout: use_delta_encoding: 1 2024-06-14T22:33:19.738 INFO:journalctl@ceph.mon.b.smithi178.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-06-14T22:33:19.738 INFO:journalctl@ceph.mon.b.smithi178.stdout: whole_key_filtering: 1 2024-06-14T22:33:19.738 INFO:journalctl@ceph.mon.b.smithi178.stdout: verify_compression: 0 2024-06-14T22:33:19.738 INFO:journalctl@ceph.mon.b.smithi178.stdout: read_amp_bytes_per_bit: 0 2024-06-14T22:33:19.738 INFO:journalctl@ceph.mon.b.smithi178.stdout: format_version: 4 2024-06-14T22:33:19.738 INFO:journalctl@ceph.mon.b.smithi178.stdout: enable_index_compression: 1 2024-06-14T22:33:19.738 INFO:journalctl@ceph.mon.b.smithi178.stdout: block_align: 0 2024-06-14T22:33:19.738 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.write_buffer_size: 33554432 2024-06-14T22:33:19.738 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_write_buffer_number: 2 2024-06-14T22:33:19.738 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compression: NoCompression 2024-06-14T22:33:19.738 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.bottommost_compression: Disabled 2024-06-14T22:33:19.738 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.prefix_extractor: nullptr 2024-06-14T22:33:19.738 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-14T22:33:19.739 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.num_levels: 7 2024-06-14T22:33:19.739 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-14T22:33:19.739 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-14T22:33:19.739 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-14T22:33:19.739 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-14T22:33:19.739 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-14T22:33:19.739 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-14T22:33:19.739 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-14T22:33:19.739 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-14T22:33:19.739 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-14T22:33:19.739 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-14T22:33:19.739 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-14T22:33:19.739 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compression_opts.level: 32767 2024-06-14T22:33:19.740 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compression_opts.strategy: 0 2024-06-14T22:33:19.740 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-14T22:33:19.740 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-14T22:33:19.740 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-14T22:33:19.740 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compression_opts.enabled: false 2024-06-14T22:33:19.740 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-14T22:33:19.740 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-14T22:33:19.740 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-14T22:33:19.740 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.target_file_size_base: 67108864 2024-06-14T22:33:19.740 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-14T22:33:19.740 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-14T22:33:19.740 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-14T22:33:19.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-14T22:33:19.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-14T22:33:19.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-14T22:33:19.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-14T22:33:19.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-14T22:33:19.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-14T22:33:19.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-14T22:33:19.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-14T22:33:19.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-14T22:33:19.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-14T22:33:19.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.arena_block_size: 4194304 2024-06-14T22:33:19.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-14T22:33:19.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-14T22:33:19.742 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-06-14T22:33:19.742 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.disable_auto_compactions: 0 2024-06-14T22:33:19.742 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-14T22:33:19.742 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-14T22:33:19.742 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-14T22:33:19.742 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-14T22:33:19.742 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-14T22:33:19.742 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-14T22:33:19.742 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-14T22:33:19.742 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-14T22:33:19.742 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-14T22:33:19.742 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-14T22:33:19.742 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.table_properties_collectors: 2024-06-14T22:33:19.743 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.inplace_update_support: 0 2024-06-14T22:33:19.743 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-14T22:33:19.743 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-14T22:33:19.743 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-14T22:33:19.743 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-14T22:33:19.743 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.bloom_locality: 0 2024-06-14T22:33:19.743 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.max_successive_merges: 0 2024-06-14T22:33:19.743 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-14T22:33:19.743 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.paranoid_file_checks: 0 2024-06-14T22:33:19.743 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.force_consistency_checks: 1 2024-06-14T22:33:19.743 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.report_bg_io_stats: 0 2024-06-14T22:33:19.744 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.ttl: 2592000 2024-06-14T22:33:19.744 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-14T22:33:19.744 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.enable_blob_files: false 2024-06-14T22:33:19.744 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.min_blob_size: 0 2024-06-14T22:33:19.744 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.blob_file_size: 268435456 2024-06-14T22:33:19.744 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-14T22:33:19.744 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-14T22:33:19.744 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-14T22:33:19.744 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-14T22:33:19.744 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-06-14T22:33:19.744 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-06-14T22:33:19.744 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718404399408200, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-06-14T22:33:19.744 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-06-14T22:33:19.745 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-06-14T22:33:19.745 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718404399408495, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1545, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1718404399, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "373e075b-7f03-4648-a8f4-9beefcf9e19d", "db_session_id": "AJZIBPCTN2RGBFTBIDFJ"}} 2024-06-14T22:33:19.745 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-06-14T22:33:19.745 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718404399408818, "job": 1, "event": "recovery_finished"} 2024-06-14T22:33:19.745 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-14T22:33:19.745 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x561905570700 2024-06-14T22:33:19.745 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: DB pointer 0x561905540000 2024-06-14T22:33:19.745 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-06-14T22:33:19.745 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-06-14T22:33:19.745 INFO:journalctl@ceph.mon.b.smithi178.stdout: ** DB Stats ** 2024-06-14T22:33:19.745 INFO:journalctl@ceph.mon.b.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-14T22:33:19.745 INFO:journalctl@ceph.mon.b.smithi178.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-14T22:33:19.746 INFO:journalctl@ceph.mon.b.smithi178.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-14T22:33:19.746 INFO:journalctl@ceph.mon.b.smithi178.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-14T22:33:19.746 INFO:journalctl@ceph.mon.b.smithi178.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-14T22:33:19.746 INFO:journalctl@ceph.mon.b.smithi178.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-06-14T22:33:19.746 INFO:journalctl@ceph.mon.b.smithi178.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-14T22:33:19.746 INFO:journalctl@ceph.mon.b.smithi178.stdout: 2024-06-14T22:33:19.746 INFO:journalctl@ceph.mon.b.smithi178.stdout: ** Compaction Stats [default] ** 2024-06-14T22:33:19.746 INFO:journalctl@ceph.mon.b.smithi178.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-14T22:33:19.746 INFO:journalctl@ceph.mon.b.smithi178.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-14T22:33:19.746 INFO:journalctl@ceph.mon.b.smithi178.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-06-14T22:33:19.746 INFO:journalctl@ceph.mon.b.smithi178.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-06-14T22:33:19.747 INFO:journalctl@ceph.mon.b.smithi178.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-06-14T22:33:19.747 INFO:journalctl@ceph.mon.b.smithi178.stdout: 2024-06-14T22:33:19.747 INFO:journalctl@ceph.mon.b.smithi178.stdout: ** Compaction Stats [default] ** 2024-06-14T22:33:19.747 INFO:journalctl@ceph.mon.b.smithi178.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-14T22:33:19.747 INFO:journalctl@ceph.mon.b.smithi178.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-14T22:33:19.747 INFO:journalctl@ceph.mon.b.smithi178.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-06-14T22:33:19.747 INFO:journalctl@ceph.mon.b.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-14T22:33:19.747 INFO:journalctl@ceph.mon.b.smithi178.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-14T22:33:19.747 INFO:journalctl@ceph.mon.b.smithi178.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-14T22:33:19.747 INFO:journalctl@ceph.mon.b.smithi178.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-14T22:33:19.747 INFO:journalctl@ceph.mon.b.smithi178.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-14T22:33:19.747 INFO:journalctl@ceph.mon.b.smithi178.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-14T22:33:19.748 INFO:journalctl@ceph.mon.b.smithi178.stdout: Cumulative compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-14T22:33:19.748 INFO:journalctl@ceph.mon.b.smithi178.stdout: Interval compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-14T22:33:19.748 INFO:journalctl@ceph.mon.b.smithi178.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-14T22:33:19.748 INFO:journalctl@ceph.mon.b.smithi178.stdout: 2024-06-14T22:33:19.748 INFO:journalctl@ceph.mon.b.smithi178.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-14T22:33:19.748 INFO:journalctl@ceph.mon.b.smithi178.stdout: 2024-06-14T22:33:19.748 INFO:journalctl@ceph.mon.b.smithi178.stdout: ** Compaction Stats [default] ** 2024-06-14T22:33:19.748 INFO:journalctl@ceph.mon.b.smithi178.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-14T22:33:19.748 INFO:journalctl@ceph.mon.b.smithi178.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-14T22:33:19.748 INFO:journalctl@ceph.mon.b.smithi178.stdout: L0 1/0 1.51 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-06-14T22:33:19.748 INFO:journalctl@ceph.mon.b.smithi178.stdout: Sum 1/0 1.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-06-14T22:33:19.748 INFO:journalctl@ceph.mon.b.smithi178.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-06-14T22:33:19.748 INFO:journalctl@ceph.mon.b.smithi178.stdout: 2024-06-14T22:33:19.749 INFO:journalctl@ceph.mon.b.smithi178.stdout: ** Compaction Stats [default] ** 2024-06-14T22:33:19.749 INFO:journalctl@ceph.mon.b.smithi178.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-14T22:33:19.749 INFO:journalctl@ceph.mon.b.smithi178.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-14T22:33:19.749 INFO:journalctl@ceph.mon.b.smithi178.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-06-14T22:33:19.749 INFO:journalctl@ceph.mon.b.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-14T22:33:19.749 INFO:journalctl@ceph.mon.b.smithi178.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-14T22:33:19.749 INFO:journalctl@ceph.mon.b.smithi178.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-14T22:33:19.749 INFO:journalctl@ceph.mon.b.smithi178.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-14T22:33:19.749 INFO:journalctl@ceph.mon.b.smithi178.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-14T22:33:19.749 INFO:journalctl@ceph.mon.b.smithi178.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-14T22:33:19.749 INFO:journalctl@ceph.mon.b.smithi178.stdout: Cumulative compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-14T22:33:19.749 INFO:journalctl@ceph.mon.b.smithi178.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-14T22:33:19.749 INFO:journalctl@ceph.mon.b.smithi178.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-14T22:33:19.750 INFO:journalctl@ceph.mon.b.smithi178.stdout: 2024-06-14T22:33:19.750 INFO:journalctl@ceph.mon.b.smithi178.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-14T22:33:19.750 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-06-14T22:33:19.750 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: using public_addr v2:172.21.15.178:0/0 -> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-06-14T22:33:19.750 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: starting mon.b rank -1 at public addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] at bind addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:33:19.750 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: mon.b@-1(???) e0 preinit fsid cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:33:19.750 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: mon.b@-1(synchronizing).mds e1 new map 2024-06-14T22:33:19.750 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: mon.b@-1(synchronizing).mds e1 print_map 2024-06-14T22:33:19.750 INFO:journalctl@ceph.mon.b.smithi178.stdout: e1 2024-06-14T22:33:19.750 INFO:journalctl@ceph.mon.b.smithi178.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-14T22:33:19.750 INFO:journalctl@ceph.mon.b.smithi178.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-14T22:33:19.751 INFO:journalctl@ceph.mon.b.smithi178.stdout: legacy client fscid: -1 2024-06-14T22:33:19.751 INFO:journalctl@ceph.mon.b.smithi178.stdout: 2024-06-14T22:33:19.751 INFO:journalctl@ceph.mon.b.smithi178.stdout: No filesystems configured 2024-06-14T22:33:19.751 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-14T22:33:19.751 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-14T22:33:19.751 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-14T22:33:19.751 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-14T22:33:19.751 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-14T22:33:19.751 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-14T22:33:19.751 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-14T22:33:19.751 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-14T22:33:19.751 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-14T22:33:19.751 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-14T22:33:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/1314704589' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-14T22:33:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: osdmap e4: 0 total, 0 up, 0 in 2024-06-14T22:33:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:33:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:33:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-14T22:33:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:19.752 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi046:[v2:172.21.15.46:3301,v1:172.21.15.46:6790]=c;smithi178:172.21.15.178=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:33:19.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: Saving service mon spec with placement smithi046:172.21.15.46=a;smithi046:[v2:172.21.15.46:3301,v1:172.21.15.46:6790]=c;smithi178:172.21.15.178=b;count:3 2024-06-14T22:33:19.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: Deploying daemon mon.b on smithi178 2024-06-14T22:33:19.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:19.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:19.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: from='client.? 172.21.15.178:0/1490751780' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-14T22:33:19.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 ceph-mon[32942]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-14T22:33:19.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 bash[32880]: cf042b827361861b30df3fec20ea522876b01fd7706b3f7846734dd486352577 2024-06-14T22:33:19.753 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:19 smithi178 systemd[1]: Started Ceph mon.b for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:33:24.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:33:24.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:24.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: mon.a calling monitor election 2024-06-14T22:33:24.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:24.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:24.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:24.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: mon.b calling monitor election 2024-06-14T22:33:24.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:24.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:24.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:24.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:24.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-06-14T22:33:24.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: monmap e2: 2 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],b=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-14T22:33:24.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: fsmap 2024-06-14T22:33:24.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: osdmap e4: 0 total, 0 up, 0 in 2024-06-14T22:33:24.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: mgrmap e13: y(active, since 40s) 2024-06-14T22:33:24.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: overall HEALTH_OK 2024-06-14T22:33:24.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:24.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:24.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:24.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-14T22:33:24.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:24.855 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:33:24.855 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":2,"fsid":"cb009720-2a9d-11ef-bc9d-c7b262605968","modified":"2024-06-14T22:33:19.429762Z","created":"2024-06-14T22:31:47.571363Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-14T22:33:24.855 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 2 2024-06-14T22:33:24.888 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:33:24.888 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:24.888 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: mon.a calling monitor election 2024-06-14T22:33:24.888 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:24.888 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:24.888 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:24.888 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: mon.b calling monitor election 2024-06-14T22:33:24.888 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:24.888 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:24.888 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:24.888 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:24.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-06-14T22:33:24.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: monmap e2: 2 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],b=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-14T22:33:24.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: fsmap 2024-06-14T22:33:24.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: osdmap e4: 0 total, 0 up, 0 in 2024-06-14T22:33:24.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: mgrmap e13: y(active, since 40s) 2024-06-14T22:33:24.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: overall HEALTH_OK 2024-06-14T22:33:24.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:24.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:24.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:24.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-14T22:33:24.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:25.751 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:25 smithi046 ceph-mon[29586]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:25.751 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:25 smithi046 ceph-mon[29586]: Deploying daemon mon.c on smithi046 2024-06-14T22:33:25.752 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:25 smithi046 ceph-mon[29586]: from='client.? 172.21.15.178:0/3078759327' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-14T22:33:25.752 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:25 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:25.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:25 smithi178 ceph-mon[32942]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:25.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:25 smithi178 ceph-mon[32942]: Deploying daemon mon.c on smithi046 2024-06-14T22:33:25.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:25 smithi178 ceph-mon[32942]: from='client.? 172.21.15.178:0/3078759327' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-14T22:33:25.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:25 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:26.518 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-14T22:33:26.518 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph mon dump -f json 2024-06-14T22:33:26.769 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.b/config 2024-06-14T22:33:27.494 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 podman[38248]: 2024-06-14 22:33:27.421812164 +0000 UTC m=+0.117161698 container create 2306f93e8c1b6d02a00989a576e5067b77bc6311ca0ce7babcd460d78debab25 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-c, org.label-schema.build-date=20240612, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=quincy-e47ff4b) 2024-06-14T22:33:27.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:27 smithi178 ceph-mon[32942]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:27.760 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[29586]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:27.761 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 podman[38248]: 2024-06-14 22:33:27.674320386 +0000 UTC m=+0.369669926 container init 2306f93e8c1b6d02a00989a576e5067b77bc6311ca0ce7babcd460d78debab25 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-c, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.build-date=20240612, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=quincy-e47ff4b, org.label-schema.schema-version=1.0) 2024-06-14T22:33:27.761 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 podman[38248]: 2024-06-14 22:33:27.680297563 +0000 UTC m=+0.375647092 container start 2306f93e8c1b6d02a00989a576e5067b77bc6311ca0ce7babcd460d78debab25 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-e47ff4b, org.label-schema.build-date=20240612, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-14T22:33:27.761 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: set uid:gid to 167:167 (ceph:ceph) 2024-06-14T22:33:27.761 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: ceph version 17.2.7-1008-ge47ff4b8 (e47ff4b87f747658fc7376cc7065fc56801eb37b) quincy (stable), process ceph-mon, pid 7 2024-06-14T22:33:27.761 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: pidfile_write: ignore empty --pid-file 2024-06-14T22:33:27.761 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: load: jerasure load: lrc 2024-06-14T22:33:27.761 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: RocksDB version: 6.15.5 2024-06-14T22:33:27.761 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-06-14T22:33:27.761 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Compile date Jun 14 2024 2024-06-14T22:33:27.761 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: DB SUMMARY 2024-06-14T22:33:27.761 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: DB Session ID: KZIXG2YVXZNRTDRZT72H 2024-06-14T22:33:27.762 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: CURRENT file: CURRENT 2024-06-14T22:33:27.762 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: IDENTITY file: IDENTITY 2024-06-14T22:33:27.762 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-06-14T22:33:27.762 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-06-14T22:33:27.762 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-06-14T22:33:27.762 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.error_if_exists: 0 2024-06-14T22:33:27.762 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.create_if_missing: 0 2024-06-14T22:33:27.762 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.paranoid_checks: 1 2024-06-14T22:33:27.762 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-14T22:33:27.762 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.env: 0x557cd5b12440 2024-06-14T22:33:27.762 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.fs: Posix File System 2024-06-14T22:33:27.762 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.info_log: 0x557cd649cf40 2024-06-14T22:33:27.763 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_file_opening_threads: 16 2024-06-14T22:33:27.763 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.statistics: (nil) 2024-06-14T22:33:27.763 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.use_fsync: 0 2024-06-14T22:33:27.763 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_log_file_size: 0 2024-06-14T22:33:27.764 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-14T22:33:27.764 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-14T22:33:27.764 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.keep_log_file_num: 1000 2024-06-14T22:33:27.765 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.recycle_log_file_num: 0 2024-06-14T22:33:27.765 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.allow_fallocate: 1 2024-06-14T22:33:27.765 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.allow_mmap_reads: 0 2024-06-14T22:33:27.765 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.allow_mmap_writes: 0 2024-06-14T22:33:27.765 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.use_direct_reads: 0 2024-06-14T22:33:27.765 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-14T22:33:27.765 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.create_missing_column_families: 0 2024-06-14T22:33:27.765 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.db_log_dir: 2024-06-14T22:33:27.765 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-06-14T22:33:27.765 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-14T22:33:27.765 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-14T22:33:27.765 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-14T22:33:27.766 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-14T22:33:27.766 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-14T22:33:27.766 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-14T22:33:27.766 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.advise_random_on_open: 1 2024-06-14T22:33:27.766 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.db_write_buffer_size: 0 2024-06-14T22:33:27.766 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.write_buffer_manager: 0x557cd655a780 2024-06-14T22:33:27.766 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-14T22:33:27.766 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-06-14T22:33:27.766 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-14T22:33:27.766 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-14T22:33:27.766 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.rate_limiter: (nil) 2024-06-14T22:33:27.766 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-14T22:33:27.767 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.wal_recovery_mode: 2 2024-06-14T22:33:27.767 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.enable_thread_tracking: 0 2024-06-14T22:33:27.767 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.enable_pipelined_write: 0 2024-06-14T22:33:27.767 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.unordered_write: 0 2024-06-14T22:33:27.767 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-14T22:33:27.767 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-14T22:33:27.767 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-14T22:33:27.767 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-14T22:33:27.767 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.row_cache: None 2024-06-14T22:33:27.767 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.wal_filter: None 2024-06-14T22:33:27.767 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-14T22:33:27.767 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.allow_ingest_behind: 0 2024-06-14T22:33:27.767 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.preserve_deletes: 0 2024-06-14T22:33:27.767 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.two_write_queues: 0 2024-06-14T22:33:27.768 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.manual_wal_flush: 0 2024-06-14T22:33:27.768 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.atomic_flush: 0 2024-06-14T22:33:27.768 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-14T22:33:27.768 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-14T22:33:27.768 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-14T22:33:27.768 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.log_readahead_size: 0 2024-06-14T22:33:27.768 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-14T22:33:27.768 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.best_efforts_recovery: 0 2024-06-14T22:33:27.768 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-14T22:33:27.768 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-14T22:33:27.768 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.allow_data_in_errors: 0 2024-06-14T22:33:27.768 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.db_host_id: __hostname__ 2024-06-14T22:33:27.769 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_background_jobs: 2 2024-06-14T22:33:27.769 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_background_compactions: -1 2024-06-14T22:33:27.769 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_subcompactions: 1 2024-06-14T22:33:27.769 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-14T22:33:27.769 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-14T22:33:27.769 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-14T22:33:27.769 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_total_wal_size: 0 2024-06-14T22:33:27.769 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-14T22:33:27.769 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-14T22:33:27.769 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-14T22:33:27.769 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-14T22:33:27.770 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_open_files: -1 2024-06-14T22:33:27.770 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.bytes_per_sync: 0 2024-06-14T22:33:27.770 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-14T22:33:27.770 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-14T22:33:27.770 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compaction_readahead_size: 0 2024-06-14T22:33:27.770 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_background_flushes: -1 2024-06-14T22:33:27.770 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Compression algorithms supported: 2024-06-14T22:33:27.770 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: kZSTD supported: 0 2024-06-14T22:33:27.770 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: kXpressCompression supported: 0 2024-06-14T22:33:27.770 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: kBZip2Compression supported: 0 2024-06-14T22:33:27.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-14T22:33:27.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: kLZ4Compression supported: 1 2024-06-14T22:33:27.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: kZlibCompression supported: 1 2024-06-14T22:33:27.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: kLZ4HCCompression supported: 1 2024-06-14T22:33:27.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: kSnappyCompression supported: 1 2024-06-14T22:33:27.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-14T22:33:27.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 2024-06-14T22:33:27.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-06-14T22:33:27.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-14T22:33:27.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.merge_operator: 2024-06-14T22:33:27.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compaction_filter: None 2024-06-14T22:33:27.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compaction_filter_factory: None 2024-06-14T22:33:27.772 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.sst_partitioner_factory: None 2024-06-14T22:33:27.772 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-14T22:33:27.772 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-14T22:33:27.772 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557cd6443a68) 2024-06-14T22:33:27.772 INFO:journalctl@ceph.mon.c.smithi046.stdout: cache_index_and_filter_blocks: 1 2024-06-14T22:33:27.772 INFO:journalctl@ceph.mon.c.smithi046.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-14T22:33:27.772 INFO:journalctl@ceph.mon.c.smithi046.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-14T22:33:27.772 INFO:journalctl@ceph.mon.c.smithi046.stdout: pin_top_level_index_and_filter: 1 2024-06-14T22:33:27.772 INFO:journalctl@ceph.mon.c.smithi046.stdout: index_type: 0 2024-06-14T22:33:27.772 INFO:journalctl@ceph.mon.c.smithi046.stdout: data_block_index_type: 0 2024-06-14T22:33:27.772 INFO:journalctl@ceph.mon.c.smithi046.stdout: index_shortening: 1 2024-06-14T22:33:27.772 INFO:journalctl@ceph.mon.c.smithi046.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-14T22:33:27.772 INFO:journalctl@ceph.mon.c.smithi046.stdout: hash_index_allow_collision: 1 2024-06-14T22:33:27.772 INFO:journalctl@ceph.mon.c.smithi046.stdout: checksum: 1 2024-06-14T22:33:27.773 INFO:journalctl@ceph.mon.c.smithi046.stdout: no_block_cache: 0 2024-06-14T22:33:27.773 INFO:journalctl@ceph.mon.c.smithi046.stdout: block_cache: 0x557cd6479350 2024-06-14T22:33:27.773 INFO:journalctl@ceph.mon.c.smithi046.stdout: block_cache_name: BinnedLRUCache 2024-06-14T22:33:27.773 INFO:journalctl@ceph.mon.c.smithi046.stdout: block_cache_options: 2024-06-14T22:33:27.773 INFO:journalctl@ceph.mon.c.smithi046.stdout: capacity : 536870912 2024-06-14T22:33:27.773 INFO:journalctl@ceph.mon.c.smithi046.stdout: num_shard_bits : 4 2024-06-14T22:33:27.773 INFO:journalctl@ceph.mon.c.smithi046.stdout: strict_capacity_limit : 0 2024-06-14T22:33:27.773 INFO:journalctl@ceph.mon.c.smithi046.stdout: high_pri_pool_ratio: 0.000 2024-06-14T22:33:27.773 INFO:journalctl@ceph.mon.c.smithi046.stdout: block_cache_compressed: (nil) 2024-06-14T22:33:27.773 INFO:journalctl@ceph.mon.c.smithi046.stdout: persistent_cache: (nil) 2024-06-14T22:33:27.773 INFO:journalctl@ceph.mon.c.smithi046.stdout: block_size: 4096 2024-06-14T22:33:27.773 INFO:journalctl@ceph.mon.c.smithi046.stdout: block_size_deviation: 10 2024-06-14T22:33:27.773 INFO:journalctl@ceph.mon.c.smithi046.stdout: block_restart_interval: 16 2024-06-14T22:33:27.773 INFO:journalctl@ceph.mon.c.smithi046.stdout: index_block_restart_interval: 1 2024-06-14T22:33:27.774 INFO:journalctl@ceph.mon.c.smithi046.stdout: metadata_block_size: 4096 2024-06-14T22:33:27.774 INFO:journalctl@ceph.mon.c.smithi046.stdout: partition_filters: 0 2024-06-14T22:33:27.774 INFO:journalctl@ceph.mon.c.smithi046.stdout: use_delta_encoding: 1 2024-06-14T22:33:27.774 INFO:journalctl@ceph.mon.c.smithi046.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-06-14T22:33:27.774 INFO:journalctl@ceph.mon.c.smithi046.stdout: whole_key_filtering: 1 2024-06-14T22:33:27.774 INFO:journalctl@ceph.mon.c.smithi046.stdout: verify_compression: 0 2024-06-14T22:33:27.774 INFO:journalctl@ceph.mon.c.smithi046.stdout: read_amp_bytes_per_bit: 0 2024-06-14T22:33:27.774 INFO:journalctl@ceph.mon.c.smithi046.stdout: format_version: 4 2024-06-14T22:33:27.774 INFO:journalctl@ceph.mon.c.smithi046.stdout: enable_index_compression: 1 2024-06-14T22:33:27.774 INFO:journalctl@ceph.mon.c.smithi046.stdout: block_align: 0 2024-06-14T22:33:27.774 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.write_buffer_size: 33554432 2024-06-14T22:33:27.774 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_write_buffer_number: 2 2024-06-14T22:33:27.775 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compression: NoCompression 2024-06-14T22:33:27.775 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.bottommost_compression: Disabled 2024-06-14T22:33:27.775 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.prefix_extractor: nullptr 2024-06-14T22:33:27.775 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-14T22:33:27.775 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.num_levels: 7 2024-06-14T22:33:27.775 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-14T22:33:27.775 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-14T22:33:27.775 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-14T22:33:27.775 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-14T22:33:27.775 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-14T22:33:27.775 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-14T22:33:27.775 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-14T22:33:27.776 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-14T22:33:27.776 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-14T22:33:27.776 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-14T22:33:27.776 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-14T22:33:27.776 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compression_opts.level: 32767 2024-06-14T22:33:27.776 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compression_opts.strategy: 0 2024-06-14T22:33:27.776 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-14T22:33:27.776 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-14T22:33:27.776 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-14T22:33:27.776 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compression_opts.enabled: false 2024-06-14T22:33:27.776 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-14T22:33:27.776 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-14T22:33:27.776 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-14T22:33:27.777 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.target_file_size_base: 67108864 2024-06-14T22:33:27.777 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-14T22:33:27.777 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-14T22:33:27.777 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-14T22:33:27.777 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-14T22:33:27.777 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-14T22:33:27.777 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-14T22:33:27.777 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-14T22:33:27.777 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-14T22:33:27.777 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-14T22:33:27.778 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-14T22:33:27.778 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-14T22:33:27.778 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-14T22:33:27.778 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-14T22:33:27.778 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.arena_block_size: 4194304 2024-06-14T22:33:27.778 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-14T22:33:27.778 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-14T22:33:27.778 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-06-14T22:33:27.778 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.disable_auto_compactions: 0 2024-06-14T22:33:27.778 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-14T22:33:27.778 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-14T22:33:27.778 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-14T22:33:27.779 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-14T22:33:27.779 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-14T22:33:27.779 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-14T22:33:27.779 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-14T22:33:27.779 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-14T22:33:27.779 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-14T22:33:27.779 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-14T22:33:27.779 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.table_properties_collectors: 2024-06-14T22:33:27.779 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.inplace_update_support: 0 2024-06-14T22:33:27.779 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-14T22:33:27.779 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-14T22:33:27.779 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-14T22:33:27.779 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-14T22:33:27.780 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.bloom_locality: 0 2024-06-14T22:33:27.780 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.max_successive_merges: 0 2024-06-14T22:33:27.780 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-14T22:33:27.780 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.paranoid_file_checks: 0 2024-06-14T22:33:27.780 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.force_consistency_checks: 1 2024-06-14T22:33:27.780 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.report_bg_io_stats: 0 2024-06-14T22:33:27.780 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.ttl: 2592000 2024-06-14T22:33:27.780 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-14T22:33:27.780 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.enable_blob_files: false 2024-06-14T22:33:27.780 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.min_blob_size: 0 2024-06-14T22:33:27.780 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.blob_file_size: 268435456 2024-06-14T22:33:27.780 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-14T22:33:27.780 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-14T22:33:27.781 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-14T22:33:27.781 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-06-14T22:33:27.781 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-06-14T22:33:27.781 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-06-14T22:33:27.781 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718404407705149, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-06-14T22:33:27.781 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-06-14T22:33:27.781 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-06-14T22:33:27.781 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718404407705469, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1670, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1718404407, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "760d7fcb-db3e-45ed-b4e7-56b6437fb31c", "db_session_id": "KZIXG2YVXZNRTDRZT72H"}} 2024-06-14T22:33:27.781 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-06-14T22:33:27.781 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: EVENT_LOG_v1 {"time_micros": 1718404407705848, "job": 1, "event": "recovery_finished"} 2024-06-14T22:33:27.781 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-14T22:33:27.781 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x557cd64ac700 2024-06-14T22:33:27.781 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: DB pointer 0x557cd647c000 2024-06-14T22:33:27.782 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-06-14T22:33:27.782 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-06-14T22:33:27.782 INFO:journalctl@ceph.mon.c.smithi046.stdout: ** DB Stats ** 2024-06-14T22:33:27.782 INFO:journalctl@ceph.mon.c.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-14T22:33:27.782 INFO:journalctl@ceph.mon.c.smithi046.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-14T22:33:27.782 INFO:journalctl@ceph.mon.c.smithi046.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-14T22:33:27.782 INFO:journalctl@ceph.mon.c.smithi046.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-14T22:33:27.782 INFO:journalctl@ceph.mon.c.smithi046.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-14T22:33:27.782 INFO:journalctl@ceph.mon.c.smithi046.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-06-14T22:33:27.782 INFO:journalctl@ceph.mon.c.smithi046.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-14T22:33:27.782 INFO:journalctl@ceph.mon.c.smithi046.stdout: 2024-06-14T22:33:27.782 INFO:journalctl@ceph.mon.c.smithi046.stdout: ** Compaction Stats [default] ** 2024-06-14T22:33:27.782 INFO:journalctl@ceph.mon.c.smithi046.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-14T22:33:27.783 INFO:journalctl@ceph.mon.c.smithi046.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-14T22:33:27.783 INFO:journalctl@ceph.mon.c.smithi046.stdout: L0 1/0 1.63 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-06-14T22:33:27.783 INFO:journalctl@ceph.mon.c.smithi046.stdout: Sum 1/0 1.63 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-06-14T22:33:27.783 INFO:journalctl@ceph.mon.c.smithi046.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-06-14T22:33:27.783 INFO:journalctl@ceph.mon.c.smithi046.stdout: 2024-06-14T22:33:27.783 INFO:journalctl@ceph.mon.c.smithi046.stdout: ** Compaction Stats [default] ** 2024-06-14T22:33:27.783 INFO:journalctl@ceph.mon.c.smithi046.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-14T22:33:27.783 INFO:journalctl@ceph.mon.c.smithi046.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-14T22:33:27.783 INFO:journalctl@ceph.mon.c.smithi046.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-06-14T22:33:27.783 INFO:journalctl@ceph.mon.c.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-14T22:33:27.783 INFO:journalctl@ceph.mon.c.smithi046.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-14T22:33:27.783 INFO:journalctl@ceph.mon.c.smithi046.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-14T22:33:27.783 INFO:journalctl@ceph.mon.c.smithi046.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-14T22:33:27.784 INFO:journalctl@ceph.mon.c.smithi046.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-14T22:33:27.784 INFO:journalctl@ceph.mon.c.smithi046.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-14T22:33:27.784 INFO:journalctl@ceph.mon.c.smithi046.stdout: Cumulative compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-14T22:33:27.784 INFO:journalctl@ceph.mon.c.smithi046.stdout: Interval compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-14T22:33:27.784 INFO:journalctl@ceph.mon.c.smithi046.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-14T22:33:27.784 INFO:journalctl@ceph.mon.c.smithi046.stdout: 2024-06-14T22:33:27.784 INFO:journalctl@ceph.mon.c.smithi046.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-14T22:33:27.784 INFO:journalctl@ceph.mon.c.smithi046.stdout: 2024-06-14T22:33:27.784 INFO:journalctl@ceph.mon.c.smithi046.stdout: ** Compaction Stats [default] ** 2024-06-14T22:33:27.784 INFO:journalctl@ceph.mon.c.smithi046.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-14T22:33:27.784 INFO:journalctl@ceph.mon.c.smithi046.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-14T22:33:27.784 INFO:journalctl@ceph.mon.c.smithi046.stdout: L0 1/0 1.63 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-06-14T22:33:27.784 INFO:journalctl@ceph.mon.c.smithi046.stdout: Sum 1/0 1.63 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-06-14T22:33:27.784 INFO:journalctl@ceph.mon.c.smithi046.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-06-14T22:33:27.785 INFO:journalctl@ceph.mon.c.smithi046.stdout: 2024-06-14T22:33:27.785 INFO:journalctl@ceph.mon.c.smithi046.stdout: ** Compaction Stats [default] ** 2024-06-14T22:33:27.785 INFO:journalctl@ceph.mon.c.smithi046.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-06-14T22:33:27.785 INFO:journalctl@ceph.mon.c.smithi046.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-14T22:33:27.785 INFO:journalctl@ceph.mon.c.smithi046.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-06-14T22:33:27.785 INFO:journalctl@ceph.mon.c.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-14T22:33:27.785 INFO:journalctl@ceph.mon.c.smithi046.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-14T22:33:27.785 INFO:journalctl@ceph.mon.c.smithi046.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-14T22:33:27.785 INFO:journalctl@ceph.mon.c.smithi046.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-14T22:33:27.785 INFO:journalctl@ceph.mon.c.smithi046.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-14T22:33:27.785 INFO:journalctl@ceph.mon.c.smithi046.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-14T22:33:27.785 INFO:journalctl@ceph.mon.c.smithi046.stdout: Cumulative compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-14T22:33:27.785 INFO:journalctl@ceph.mon.c.smithi046.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-14T22:33:27.786 INFO:journalctl@ceph.mon.c.smithi046.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-14T22:33:27.786 INFO:journalctl@ceph.mon.c.smithi046.stdout: 2024-06-14T22:33:27.786 INFO:journalctl@ceph.mon.c.smithi046.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-14T22:33:27.786 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-06-14T22:33:27.786 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: using public_addrv [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] 2024-06-14T22:33:27.786 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: starting mon.c rank -1 at public addrs [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] at bind addrs [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:33:27.786 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: mon.c@-1(???) e0 preinit fsid cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:33:27.786 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: mon.c@-1(synchronizing).mds e1 new map 2024-06-14T22:33:27.786 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: mon.c@-1(synchronizing).mds e1 print_map 2024-06-14T22:33:27.786 INFO:journalctl@ceph.mon.c.smithi046.stdout: e1 2024-06-14T22:33:27.786 INFO:journalctl@ceph.mon.c.smithi046.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-14T22:33:27.786 INFO:journalctl@ceph.mon.c.smithi046.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-06-14T22:33:27.787 INFO:journalctl@ceph.mon.c.smithi046.stdout: legacy client fscid: -1 2024-06-14T22:33:27.787 INFO:journalctl@ceph.mon.c.smithi046.stdout: 2024-06-14T22:33:27.787 INFO:journalctl@ceph.mon.c.smithi046.stdout: No filesystems configured 2024-06-14T22:33:27.787 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-14T22:33:27.787 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-14T22:33:27.787 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-14T22:33:27.787 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-14T22:33:27.787 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-14T22:33:27.787 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-14T22:33:27.787 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-14T22:33:27.787 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-14T22:33:27.787 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-14T22:33:27.787 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-14T22:33:27.788 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:27.788 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/1314704589' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-14T22:33:27.788 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: osdmap e4: 0 total, 0 up, 0 in 2024-06-14T22:33:27.788 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:27.788 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:27.788 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:33:27.788 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:27.788 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:33:27.788 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:27.788 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-14T22:33:27.788 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:27.788 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi046:[v2:172.21.15.46:3301,v1:172.21.15.46:6790]=c;smithi178:172.21.15.178=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:33:27.788 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: Saving service mon spec with placement smithi046:172.21.15.46=a;smithi046:[v2:172.21.15.46:3301,v1:172.21.15.46:6790]=c;smithi178:172.21.15.178=b;count:3 2024-06-14T22:33:27.789 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: Deploying daemon mon.b on smithi178 2024-06-14T22:33:27.789 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:27.789 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:27.789 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='client.? 172.21.15.178:0/1490751780' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-14T22:33:27.789 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:33:27.789 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:27.789 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: mon.a calling monitor election 2024-06-14T22:33:27.789 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:27.789 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:27.789 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:27.789 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: mon.b calling monitor election 2024-06-14T22:33:27.789 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:27.790 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:27.790 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:27.790 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:27.790 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-06-14T22:33:27.790 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: monmap e2: 2 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],b=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-14T22:33:27.790 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: fsmap 2024-06-14T22:33:27.790 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: osdmap e4: 0 total, 0 up, 0 in 2024-06-14T22:33:27.790 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: mgrmap e13: y(active, since 40s) 2024-06-14T22:33:27.790 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: overall HEALTH_OK 2024-06-14T22:33:27.790 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:27.790 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:27.790 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:27.791 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-14T22:33:27.791 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:27.791 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:27.791 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: Deploying daemon mon.c on smithi046 2024-06-14T22:33:27.791 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='client.? 172.21.15.178:0/3078759327' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-14T22:33:27.791 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:27.791 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:27.791 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 ceph-mon[38286]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-14T22:33:27.791 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 bash[38248]: 2306f93e8c1b6d02a00989a576e5067b77bc6311ca0ce7babcd460d78debab25 2024-06-14T22:33:27.791 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:27 smithi046 systemd[1]: Started Ceph mon.c for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:33:33.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:33:33.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: mon.a calling monitor election 2024-06-14T22:33:33.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: mon.b calling monitor election 2024-06-14T22:33:33.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:33.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:33.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:33.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:33.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:33.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:33.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:33.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:33.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:33.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:33.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-06-14T22:33:33.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: monmap e3: 3 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],b=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0],c=[v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-14T22:33:33.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: fsmap 2024-06-14T22:33:33.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: osdmap e4: 0 total, 0 up, 0 in 2024-06-14T22:33:33.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: mgrmap e13: y(active, since 48s) 2024-06-14T22:33:33.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-06-14T22:33:33.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-06-14T22:33:33.195 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-06-14T22:33:33.195 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: mon.c (rank 2) addr [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] is down (out of quorum) 2024-06-14T22:33:33.195 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:33.195 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:33.195 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:33.195 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:32 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:33.274 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:33:33.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: mon.a calling monitor election 2024-06-14T22:33:33.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: mon.b calling monitor election 2024-06-14T22:33:33.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:33.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:33.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:33.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:33.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:33.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:33.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:33.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:33.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:33.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:33.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-06-14T22:33:33.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: monmap e3: 3 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],b=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0],c=[v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-14T22:33:33.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: fsmap 2024-06-14T22:33:33.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: osdmap e4: 0 total, 0 up, 0 in 2024-06-14T22:33:33.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: mgrmap e13: y(active, since 48s) 2024-06-14T22:33:33.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-06-14T22:33:33.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-06-14T22:33:33.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-06-14T22:33:33.277 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: mon.c (rank 2) addr [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] is down (out of quorum) 2024-06-14T22:33:33.277 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:33.277 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:33.277 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:33.277 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:32 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:34.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:33 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:33:34.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:33 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:34.274 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:33 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:33:34.274 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:33 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:34.511 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-06-14T22:33:34.511 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":3,"fsid":"cb009720-2a9d-11ef-bc9d-c7b262605968","modified":"2024-06-14T22:33:27.726518Z","created":"2024-06-14T22:31:47.571363Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3301","nonce":0},{"type":"v1","addr":"172.21.15.46:6790","nonce":0}]},"addr":"172.21.15.46:6790/0","public_addr":"172.21.15.46:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-14T22:33:34.512 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 3 2024-06-14T22:33:34.984 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:33:34.984 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: mon.a calling monitor election 2024-06-14T22:33:34.984 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: mon.b calling monitor election 2024-06-14T22:33:34.984 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:33:34.985 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:34.985 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:34.985 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:34.985 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:34.985 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:34.985 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:34.985 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:34.985 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:34.985 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:34.985 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-06-14T22:33:34.985 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: monmap e3: 3 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],b=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0],c=[v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-14T22:33:34.985 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: fsmap 2024-06-14T22:33:34.985 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: osdmap e4: 0 total, 0 up, 0 in 2024-06-14T22:33:34.986 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: mgrmap e13: y(active, since 48s) 2024-06-14T22:33:34.986 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-06-14T22:33:34.986 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-06-14T22:33:34.986 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-06-14T22:33:34.986 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: mon.c (rank 2) addr [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] is down (out of quorum) 2024-06-14T22:33:34.986 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:34.986 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:34.986 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:34.986 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:34.986 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:33:34.986 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:35.090 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-14T22:33:35.091 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph config generate-minimal-conf 2024-06-14T22:33:35.433 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:33:35.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[29586]: mon.c calling monitor election 2024-06-14T22:33:35.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[29586]: mon.c calling monitor election 2024-06-14T22:33:35.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[29586]: mon.b calling monitor election 2024-06-14T22:33:35.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[29586]: mon.a calling monitor election 2024-06-14T22:33:35.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[29586]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-06-14T22:33:35.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[29586]: monmap e3: 3 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],b=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0],c=[v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-14T22:33:35.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[29586]: fsmap 2024-06-14T22:33:35.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[29586]: osdmap e4: 0 total, 0 up, 0 in 2024-06-14T22:33:35.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[29586]: mgrmap e13: y(active, since 50s) 2024-06-14T22:33:35.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[29586]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-06-14T22:33:35.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[29586]: Cluster is now healthy 2024-06-14T22:33:35.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[29586]: overall HEALTH_OK 2024-06-14T22:33:35.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[38286]: mon.c calling monitor election 2024-06-14T22:33:35.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[38286]: mon.c calling monitor election 2024-06-14T22:33:35.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[38286]: mon.b calling monitor election 2024-06-14T22:33:35.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[38286]: mon.a calling monitor election 2024-06-14T22:33:35.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[38286]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-06-14T22:33:35.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[38286]: monmap e3: 3 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],b=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0],c=[v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-14T22:33:35.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[38286]: fsmap 2024-06-14T22:33:35.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[38286]: osdmap e4: 0 total, 0 up, 0 in 2024-06-14T22:33:35.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[38286]: mgrmap e13: y(active, since 50s) 2024-06-14T22:33:35.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[38286]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-06-14T22:33:35.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[38286]: Cluster is now healthy 2024-06-14T22:33:35.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:35 smithi046 ceph-mon[38286]: overall HEALTH_OK 2024-06-14T22:33:35.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:35 smithi178 ceph-mon[32942]: mon.c calling monitor election 2024-06-14T22:33:35.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:35 smithi178 ceph-mon[32942]: mon.c calling monitor election 2024-06-14T22:33:35.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:35 smithi178 ceph-mon[32942]: mon.b calling monitor election 2024-06-14T22:33:35.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:35 smithi178 ceph-mon[32942]: mon.a calling monitor election 2024-06-14T22:33:35.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:35 smithi178 ceph-mon[32942]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-06-14T22:33:35.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:35 smithi178 ceph-mon[32942]: monmap e3: 3 mons at {a=[v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0],b=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0],c=[v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-06-14T22:33:35.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:35 smithi178 ceph-mon[32942]: fsmap 2024-06-14T22:33:35.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:35 smithi178 ceph-mon[32942]: osdmap e4: 0 total, 0 up, 0 in 2024-06-14T22:33:35.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:35 smithi178 ceph-mon[32942]: mgrmap e13: y(active, since 50s) 2024-06-14T22:33:35.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:35 smithi178 ceph-mon[32942]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-06-14T22:33:35.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:35 smithi178 ceph-mon[32942]: Cluster is now healthy 2024-06-14T22:33:35.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:35 smithi178 ceph-mon[32942]: overall HEALTH_OK 2024-06-14T22:33:36.684 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:36 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:36.684 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:36 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:36.684 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:36 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:36.684 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:36 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-14T22:33:36.684 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:36 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-14T22:33:36.684 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:36 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:36.684 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:36 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:36.684 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:36 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:36.684 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:36 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:36.685 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:36 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-14T22:33:36.685 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:36 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-14T22:33:36.685 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:36 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:36.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:36 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:33:36.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:36 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:36.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:36 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:36.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:36 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-14T22:33:36.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:36 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-14T22:33:36.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:36 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:37.686 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:37 smithi046 ceph-mon[29586]: Reconfiguring mon.c (monmap changed)... 2024-06-14T22:33:37.687 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:37 smithi046 ceph-mon[29586]: Reconfiguring daemon mon.c on smithi046 2024-06-14T22:33:37.687 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:37 smithi046 ceph-mon[29586]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:37.687 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:37 smithi046 ceph-mon[38286]: Reconfiguring mon.c (monmap changed)... 2024-06-14T22:33:37.687 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:37 smithi046 ceph-mon[38286]: Reconfiguring daemon mon.c on smithi046 2024-06-14T22:33:37.687 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:37 smithi046 ceph-mon[38286]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:37.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:37 smithi178 ceph-mon[32942]: Reconfiguring mon.c (monmap changed)... 2024-06-14T22:33:37.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:37 smithi178 ceph-mon[32942]: Reconfiguring daemon mon.c on smithi046 2024-06-14T22:33:37.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:37 smithi178 ceph-mon[32942]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:38.508 INFO:teuthology.orchestra.run.smithi046.stdout:# minimal ceph.conf for cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:33:38.508 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-06-14T22:33:38.508 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = cb009720-2a9d-11ef-bc9d-c7b262605968 2024-06-14T22:33:38.508 INFO:teuthology.orchestra.run.smithi046.stdout: mon_host = [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] [v2:172.21.15.46:3301/0,v1:172.21.15.46:6790/0] 2024-06-14T22:33:39.015 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-14T22:33:39.016 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-06-14T22:33:39.016 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.conf 2024-06-14T22:33:39.045 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-06-14T22:33:39.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-14T22:33:39.113 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-14T22:33:39.113 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.conf 2024-06-14T22:33:39.142 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-14T22:33:39.142 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-14T22:33:39.208 INFO:tasks.cephadm:Adding mgr.y on smithi046 2024-06-14T22:33:39.208 INFO:tasks.cephadm:Adding mgr.x on smithi178 2024-06-14T22:33:39.209 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph orch apply mgr '2;smithi046=y;smithi178=x' 2024-06-14T22:33:39.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:39 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:39.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:39 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:39.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:39 smithi046 ceph-mon[29586]: Reconfiguring mon.b (monmap changed)... 2024-06-14T22:33:39.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:39 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-14T22:33:39.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:39 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-14T22:33:39.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:39 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:39.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:39 smithi046 ceph-mon[29586]: Reconfiguring daemon mon.b on smithi178 2024-06-14T22:33:39.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:39 smithi046 ceph-mon[29586]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:39.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:39 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/966533313' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:39.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:39 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:39.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:39 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:39.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:39 smithi046 ceph-mon[38286]: Reconfiguring mon.b (monmap changed)... 2024-06-14T22:33:39.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:39 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-14T22:33:39.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:39 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-14T22:33:39.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:39 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:39.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:39 smithi046 ceph-mon[38286]: Reconfiguring daemon mon.b on smithi178 2024-06-14T22:33:39.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:39 smithi046 ceph-mon[38286]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:39.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:39 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/966533313' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:39.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:39 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:39.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:39 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:39.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:39 smithi178 ceph-mon[32942]: Reconfiguring mon.b (monmap changed)... 2024-06-14T22:33:39.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:39 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-14T22:33:39.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:39 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-14T22:33:39.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:39 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:39.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:39 smithi178 ceph-mon[32942]: Reconfiguring daemon mon.b on smithi178 2024-06-14T22:33:39.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:39 smithi178 ceph-mon[32942]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:39.526 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:39 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/966533313' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:39.595 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.b/config 2024-06-14T22:33:40.624 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:40 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:40.624 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:40 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:40.624 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:40 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:33:40.625 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:40 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:40.625 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:40 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:33:40.625 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:40 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:40.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:40 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:40.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:40 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:40.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:40 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:33:40.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:40 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:40.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:40 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:33:40.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:40 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:40.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:40 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:40.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:40 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:40.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:40 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:33:40.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:40 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:40.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:40 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:33:40.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:40 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:41.261 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mgr update... 2024-06-14T22:33:41.820 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:41 smithi178 ceph-mon[32942]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:41.820 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:41 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:41.820 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:41 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:33:41.821 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:41 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:41.821 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:41 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:33:41.821 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:41 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:41.821 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:41 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-14T22:33:41.821 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:41 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-14T22:33:41.821 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:41 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-14T22:33:41.821 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:41 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:41.845 DEBUG:teuthology.orchestra.run.smithi178:mgr.x> sudo journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mgr.x.service 2024-06-14T22:33:41.848 INFO:tasks.cephadm:Deploying OSDs... 2024-06-14T22:33:41.848 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-06-14T22:33:41.848 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2024-06-14T22:33:41.865 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-14T22:33:41.866 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2024-06-14T22:33:41.921 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-14T22:33:41.921 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-14T22:33:41.921 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-06-14T22:33:41.922 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-14T22:33:41.922 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-06-14T22:33:41.922 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-06-14 22:33:27.141919984 +0000 2024-06-14T22:33:41.922 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-06-14 22:28:13.451756004 +0000 2024-06-14T22:33:41.922 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-06-14 22:28:13.451756004 +0000 2024-06-14T22:33:41.922 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-06-14 22:28:13.451756004 +0000 2024-06-14T22:33:41.922 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-14T22:33:41.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[29586]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:41.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:41.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:33:41.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:41.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:33:41.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:41.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-14T22:33:41.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-14T22:33:41.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-14T22:33:41.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:41.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[38286]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:41.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:41.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:33:41.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:41.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:33:41.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:41.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-14T22:33:41.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-14T22:33:41.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-14T22:33:41.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:41 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:41.967 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-06-14T22:33:41.968 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-06-14T22:33:41.968 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.00010509 s, 4.9 MB/s 2024-06-14T22:33:41.969 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-14T22:33:42.026 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2024-06-14T22:33:42.081 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-14T22:33:42.081 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-14T22:33:42.082 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-06-14T22:33:42.082 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-14T22:33:42.082 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-06-14T22:33:42.082 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-06-14 22:33:27.141919984 +0000 2024-06-14T22:33:42.082 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-06-14 22:28:13.786752876 +0000 2024-06-14T22:33:42.082 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-06-14 22:28:13.786752876 +0000 2024-06-14T22:33:42.082 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-06-14 22:28:13.786752876 +0000 2024-06-14T22:33:42.082 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-14T22:33:42.145 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-06-14T22:33:42.145 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-06-14T22:33:42.146 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000107047 s, 4.8 MB/s 2024-06-14T22:33:42.147 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-14T22:33:42.204 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2024-06-14T22:33:42.261 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-14T22:33:42.261 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-14T22:33:42.261 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-06-14T22:33:42.261 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-14T22:33:42.262 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-06-14T22:33:42.262 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-06-14 22:33:27.141919984 +0000 2024-06-14T22:33:42.262 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-06-14 22:28:14.132749645 +0000 2024-06-14T22:33:42.262 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-06-14 22:28:14.132749645 +0000 2024-06-14T22:33:42.262 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-06-14 22:28:14.132749645 +0000 2024-06-14T22:33:42.262 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-14T22:33:42.326 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-06-14T22:33:42.326 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-06-14T22:33:42.326 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000109946 s, 4.7 MB/s 2024-06-14T22:33:42.327 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-14T22:33:42.385 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2024-06-14T22:33:42.441 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-14T22:33:42.442 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-14T22:33:42.442 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-06-14T22:33:42.442 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-14T22:33:42.442 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-06-14T22:33:42.442 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-06-14 22:33:27.142919974 +0000 2024-06-14T22:33:42.442 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-06-14 22:28:14.452746657 +0000 2024-06-14T22:33:42.442 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-06-14 22:28:14.452746657 +0000 2024-06-14T22:33:42.442 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-06-14 22:28:14.452746657 +0000 2024-06-14T22:33:42.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-14T22:33:42.505 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-06-14T22:33:42.505 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-06-14T22:33:42.505 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000111492 s, 4.6 MB/s 2024-06-14T22:33:42.507 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-14T22:33:42.563 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-14T22:33:42.563 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2024-06-14T22:33:42.580 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-14T22:33:42.580 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2024-06-14T22:33:42.635 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-14T22:33:42.635 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-14T22:33:42.635 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-06-14T22:33:42.635 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-14T22:33:42.635 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-06-14T22:33:42.635 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-06-14 22:33:18.844066047 +0000 2024-06-14T22:33:42.635 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-06-14 22:28:03.677449094 +0000 2024-06-14T22:33:42.635 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-06-14 22:28:03.677449094 +0000 2024-06-14T22:33:42.635 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-06-14 22:28:03.677449094 +0000 2024-06-14T22:33:42.635 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-14T22:33:42.698 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-06-14T22:33:42.698 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-06-14T22:33:42.698 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000110951 s, 4.6 MB/s 2024-06-14T22:33:42.699 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-14T22:33:42.756 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2024-06-14T22:33:42.810 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-14T22:33:42.811 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-14T22:33:42.811 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-06-14T22:33:42.811 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-14T22:33:42.811 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-06-14T22:33:42.811 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-06-14 22:33:18.845066030 +0000 2024-06-14T22:33:42.811 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-06-14 22:28:04.008443396 +0000 2024-06-14T22:33:42.811 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-06-14 22:28:04.008443396 +0000 2024-06-14T22:33:42.811 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-06-14 22:28:04.008443396 +0000 2024-06-14T22:33:42.811 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-14T22:33:42.871 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:42 smithi178 ceph-mon[32942]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi046=y;smithi178=x", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:33:42.871 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:42 smithi178 ceph-mon[32942]: Saving service mgr spec with placement smithi046=y;smithi178=x;count:2 2024-06-14T22:33:42.871 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:42 smithi178 ceph-mon[32942]: Deploying daemon mgr.x on smithi178 2024-06-14T22:33:42.873 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-06-14T22:33:42.873 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-06-14T22:33:42.873 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000114213 s, 4.5 MB/s 2024-06-14T22:33:42.874 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-14T22:33:42.930 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2024-06-14T22:33:42.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:42 smithi046 ceph-mon[29586]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi046=y;smithi178=x", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:33:42.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:42 smithi046 ceph-mon[29586]: Saving service mgr spec with placement smithi046=y;smithi178=x;count:2 2024-06-14T22:33:42.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:42 smithi046 ceph-mon[29586]: Deploying daemon mgr.x on smithi178 2024-06-14T22:33:42.942 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:42 smithi046 ceph-mon[38286]: from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi046=y;smithi178=x", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:33:42.942 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:42 smithi046 ceph-mon[38286]: Saving service mgr spec with placement smithi046=y;smithi178=x;count:2 2024-06-14T22:33:42.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:42 smithi046 ceph-mon[38286]: Deploying daemon mgr.x on smithi178 2024-06-14T22:33:42.987 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-14T22:33:42.987 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-14T22:33:42.987 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-06-14T22:33:42.987 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-14T22:33:42.987 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-06-14T22:33:42.987 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-06-14 22:33:18.845066030 +0000 2024-06-14T22:33:42.987 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-06-14 22:28:04.322437991 +0000 2024-06-14T22:33:42.987 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-06-14 22:28:04.322437991 +0000 2024-06-14T22:33:42.987 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-06-14 22:28:04.322437991 +0000 2024-06-14T22:33:42.987 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-14T22:33:43.186 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-06-14T22:33:43.186 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-06-14T22:33:43.186 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 9.2766e-05 s, 5.5 MB/s 2024-06-14T22:33:43.186 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-14T22:33:43.202 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2024-06-14T22:33:43.263 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-14T22:33:43.264 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-14T22:33:43.264 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-06-14T22:33:43.264 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-14T22:33:43.264 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-06-14T22:33:43.264 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-06-14 22:33:43.145654413 +0000 2024-06-14T22:33:43.264 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-06-14 22:28:04.641432499 +0000 2024-06-14T22:33:43.264 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-06-14 22:28:04.641432499 +0000 2024-06-14T22:33:43.264 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-06-14 22:28:04.641432499 +0000 2024-06-14T22:33:43.265 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-14T22:33:43.363 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-06-14T22:33:43.364 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-06-14T22:33:43.364 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 9.2569e-05 s, 5.5 MB/s 2024-06-14T22:33:43.364 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-14T22:33:43.383 INFO:tasks.cephadm:Deploying osd.0 on smithi046 with /dev/vg_nvme/lv_4... 2024-06-14T22:33:43.383 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-14T22:33:43.525 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:43 smithi178 systemd[1]: Starting Ceph mgr.x for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:33:43.653 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:33:43.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:43 smithi046 ceph-mon[29586]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:43.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:43 smithi046 ceph-mon[38286]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:43.859 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:43 smithi178 ceph-mon[32942]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:43.859 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:43 smithi178 podman[34979]: 2024-06-14 22:33:43.505770039 +0000 UTC m=+0.017045077 image pull 37f9eedc23c32ee6d04590e8a81233336d2c14cd278f52a2a2746a97e6ef2d95 quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580 2024-06-14T22:33:43.859 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:43 smithi178 podman[34979]: 2024-06-14 22:33:43.609506437 +0000 UTC m=+0.120781479 container create c1f205eb48ebbd5ba3e5787678cdab417bfc3bec61c39bceecfc0fdaa2358902 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240612, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=quincy-e47ff4b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0) 2024-06-14T22:33:43.859 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:43 smithi178 podman[34979]: 2024-06-14 22:33:43.837080396 +0000 UTC m=+0.348355437 container init c1f205eb48ebbd5ba3e5787678cdab417bfc3bec61c39bceecfc0fdaa2358902 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x, org.label-schema.build-date=20240612, ceph=True, CEPH_POINT_RELEASE=, RELEASE=quincy-e47ff4b, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-06-14T22:33:43.859 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:43 smithi178 podman[34979]: 2024-06-14 22:33:43.842719443 +0000 UTC m=+0.353994475 container start c1f205eb48ebbd5ba3e5787678cdab417bfc3bec61c39bceecfc0fdaa2358902 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=quincy-e47ff4b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612, org.label-schema.license=GPLv2, ceph=True) 2024-06-14T22:33:44.163 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:43 smithi178 ceph-mgr[35018]: -- 172.21.15.178:0/1349282528 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558b92f6a9c0 con 0x558b92f77000 2024-06-14T22:33:44.163 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:43 smithi178 bash[34979]: c1f205eb48ebbd5ba3e5787678cdab417bfc3bec61c39bceecfc0fdaa2358902 2024-06-14T22:33:44.163 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:43 smithi178 systemd[1]: Started Ceph mgr.x for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:33:44.163 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:43 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:43.963+0000 7fe4824d2ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-14T22:33:44.163 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:44.096+0000 7fe4824d2ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-14T22:33:44.163 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:44.166+0000 7fe4824d2ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-14T22:33:44.772 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:44.440+0000 7fe4824d2ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-14T22:33:44.773 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:44.498+0000 7fe4824d2ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-14T22:33:45.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:45.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:45.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:45.026 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:45.026 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:33:45.026 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-mon[32942]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:45.026 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:45.026 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:44.816+0000 7fe4824d2ec0 -1 mgr[py] Module not found: 'k8sevents' 2024-06-14T22:33:45.026 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:44.816+0000 7fe4824d2ec0 -1 mgr[py] Traceback (most recent call last): 2024-06-14T22:33:45.026 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: File "/usr/share/ceph/mgr/k8sevents/__init__.py", line 1, in 2024-06-14T22:33:45.026 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: from .module import Module 2024-06-14T22:33:45.026 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: File "/usr/share/ceph/mgr/k8sevents/module.py", line 70, in 2024-06-14T22:33:45.026 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: from kubernetes.client.models.v1_event import V1Event 2024-06-14T22:33:45.026 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: ModuleNotFoundError: No module named 'kubernetes.client.models.v1_event' 2024-06-14T22:33:45.027 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:45.027 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:44.817+0000 7fe4824d2ec0 -1 mgr[py] Class not found in module 'k8sevents' 2024-06-14T22:33:45.027 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:44.817+0000 7fe4824d2ec0 -1 mgr[py] Error loading module 'k8sevents': (2) No such file or directory 2024-06-14T22:33:45.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:44 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:45.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:44 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:45.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:44 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:45.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:44 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:45.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:44 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:33:45.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:44 smithi046 ceph-mon[29586]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:45.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:44 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:45.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:44 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:45.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:44 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:45.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:44 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:45.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:44 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:45.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:44 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:33:45.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:44 smithi046 ceph-mon[38286]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:45.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:44 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:45.474 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-14T22:33:45.557 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-14T22:33:45.567 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-06-14T22:33:45.567 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-06-14T22:33:45.567 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00859293 s, 1.2 GB/s 2024-06-14T22:33:45.568 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-06-14T22:33:45.774 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:45 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:45.446+0000 7fe4824d2ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-14T22:33:45.774 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:45 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:45.556+0000 7fe4824d2ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-14T22:33:45.775 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:45 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:45.610+0000 7fe4824d2ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-14T22:33:46.136 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:45 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:45.800+0000 7fe4824d2ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-14T22:33:46.137 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:45 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:45.859+0000 7fe4824d2ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-14T22:33:46.137 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:45 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:45.923+0000 7fe4824d2ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-14T22:33:46.137 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:46 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:46.047+0000 7fe4824d2ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-14T22:33:46.254 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:46.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:46.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:46.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:33:46.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:46.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[29586]: Reconfiguring mgr.y (unknown last config time)... 2024-06-14T22:33:46.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-14T22:33:46.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-14T22:33:46.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:46.255 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[29586]: Reconfiguring daemon mgr.y on smithi046 2024-06-14T22:33:46.255 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:46.256 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:46.256 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:46.256 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:33:46.256 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:46.256 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[38286]: Reconfiguring mgr.y (unknown last config time)... 2024-06-14T22:33:46.256 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-14T22:33:46.256 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-14T22:33:46.256 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:46.256 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:46 smithi046 ceph-mon[38286]: Reconfiguring daemon mgr.y on smithi046 2024-06-14T22:33:46.257 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_4 2024-06-14T22:33:46.453 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:46 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:46.454 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:46 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:46.454 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:46 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:46.454 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:46 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:33:46.454 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:46 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:46.454 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:46 smithi178 ceph-mon[32942]: Reconfiguring mgr.y (unknown last config time)... 2024-06-14T22:33:46.454 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:46 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-14T22:33:46.454 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:46 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-14T22:33:46.454 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:46 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:46.454 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:46 smithi178 ceph-mon[32942]: Reconfiguring daemon mgr.y on smithi046 2024-06-14T22:33:46.744 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:46 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:46.457+0000 7fe4824d2ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-14T22:33:46.744 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:46 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:46.748+0000 7fe4824d2ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-14T22:33:47.024 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:46 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:46.813+0000 7fe4824d2ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-14T22:33:47.024 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:46 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:46.940+0000 7fe4824d2ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-14T22:33:47.066 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:33:47.430 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:47 smithi178 ceph-mon[32942]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:47.430 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:47 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:47.431 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:47 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:47.431 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:47 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:33:47.431 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:47 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:47.195+0000 7fe4824d2ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-14T22:33:47.431 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:47 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:47.254+0000 7fe4824d2ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-14T22:33:47.431 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:47 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:47.434+0000 7fe4824d2ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-14T22:33:47.442 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:47 smithi046 ceph-mon[38286]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:47.442 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:47 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:47.442 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:47 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:47.442 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:47 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:33:47.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:47 smithi046 ceph-mon[29586]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:47.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:47 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:47.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:47 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:47.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:47 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:33:47.774 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:47 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:47.497+0000 7fe4824d2ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-14T22:33:47.774 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:47 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:47.583+0000 7fe4824d2ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-14T22:33:47.774 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:47 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:47.641+0000 7fe4824d2ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-14T22:33:48.321 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:48 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:48.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:48 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:33:48.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:48 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:48.391 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:48 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:48.391 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:48 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:33:48.391 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:48 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:48.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:48 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:48.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:48 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:33:48.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:48 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:33:48.525 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:48 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:48.163+0000 7fe4824d2ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-14T22:33:48.525 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:48 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:48.301+0000 7fe4824d2ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-14T22:33:48.525 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:33:48 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:33:48.301+0000 7fe4824d2ec0 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: k8sevents 2024-06-14T22:33:49.412 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[29586]: Failed to load ceph-mgr modules: k8sevents 2024-06-14T22:33:49.412 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[29586]: Standby manager daemon x started 2024-06-14T22:33:49.412 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[29586]: from='mgr.? 172.21.15.178:0/3175483553' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-14T22:33:49.412 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[29586]: from='mgr.? 172.21.15.178:0/3175483553' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-14T22:33:49.413 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[29586]: from='mgr.? 172.21.15.178:0/3175483553' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-14T22:33:49.413 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[29586]: from='mgr.? 172.21.15.178:0/3175483553' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-14T22:33:49.413 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[29586]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:49.413 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:33:49.413 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:33:49.413 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:49.413 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[38286]: Failed to load ceph-mgr modules: k8sevents 2024-06-14T22:33:49.413 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[38286]: Standby manager daemon x started 2024-06-14T22:33:49.414 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[38286]: from='mgr.? 172.21.15.178:0/3175483553' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-14T22:33:49.414 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[38286]: from='mgr.? 172.21.15.178:0/3175483553' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-14T22:33:49.414 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[38286]: from='mgr.? 172.21.15.178:0/3175483553' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-14T22:33:49.414 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[38286]: from='mgr.? 172.21.15.178:0/3175483553' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-14T22:33:49.414 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[38286]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:49.414 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:33:49.414 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:33:49.414 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:49 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:49.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:49 smithi178 ceph-mon[32942]: Failed to load ceph-mgr modules: k8sevents 2024-06-14T22:33:49.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:49 smithi178 ceph-mon[32942]: Standby manager daemon x started 2024-06-14T22:33:49.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:49 smithi178 ceph-mon[32942]: from='mgr.? 172.21.15.178:0/3175483553' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-14T22:33:49.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:49 smithi178 ceph-mon[32942]: from='mgr.? 172.21.15.178:0/3175483553' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-14T22:33:49.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:49 smithi178 ceph-mon[32942]: from='mgr.? 172.21.15.178:0/3175483553' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-14T22:33:49.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:49 smithi178 ceph-mon[32942]: from='mgr.? 172.21.15.178:0/3175483553' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-14T22:33:49.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:49 smithi178 ceph-mon[32942]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:49.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:49 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:33:49.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:49 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:33:49.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:49 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:50.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:50 smithi046 ceph-mon[29586]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:33:50.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:50 smithi046 ceph-mon[29586]: mgrmap e14: y(active, since 64s), standbys: x 2024-06-14T22:33:50.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:50 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-14T22:33:50.411 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:50 smithi046 ceph-mon[38286]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:33:50.411 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:50 smithi046 ceph-mon[38286]: mgrmap e14: y(active, since 64s), standbys: x 2024-06-14T22:33:50.411 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:50 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-14T22:33:50.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:50 smithi178 ceph-mon[32942]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:33:50.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:50 smithi178 ceph-mon[32942]: mgrmap e14: y(active, since 64s), standbys: x 2024-06-14T22:33:50.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:50 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-14T22:33:51.402 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:51 smithi046 ceph-mon[29586]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:51.403 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:51 smithi046 ceph-mon[38286]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:51.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:51 smithi178 ceph-mon[32942]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:52.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:52 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/4240190615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33945598-f864-40e6-904e-1bff72ec8b04"}]: dispatch 2024-06-14T22:33:52.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:52 smithi046 ceph-mon[29586]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33945598-f864-40e6-904e-1bff72ec8b04"}]: dispatch 2024-06-14T22:33:52.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:52 smithi046 ceph-mon[29586]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33945598-f864-40e6-904e-1bff72ec8b04"}]': finished 2024-06-14T22:33:52.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:52 smithi046 ceph-mon[29586]: osdmap e5: 1 total, 0 up, 1 in 2024-06-14T22:33:52.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:52 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-14T22:33:52.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:52 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/3236510873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:33:52.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:52 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/4240190615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33945598-f864-40e6-904e-1bff72ec8b04"}]: dispatch 2024-06-14T22:33:52.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:52 smithi046 ceph-mon[38286]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33945598-f864-40e6-904e-1bff72ec8b04"}]: dispatch 2024-06-14T22:33:52.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:52 smithi046 ceph-mon[38286]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33945598-f864-40e6-904e-1bff72ec8b04"}]': finished 2024-06-14T22:33:52.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:52 smithi046 ceph-mon[38286]: osdmap e5: 1 total, 0 up, 1 in 2024-06-14T22:33:52.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:52 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-14T22:33:52.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:52 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/3236510873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:33:52.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:52 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/4240190615' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33945598-f864-40e6-904e-1bff72ec8b04"}]: dispatch 2024-06-14T22:33:52.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:52 smithi178 ceph-mon[32942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "33945598-f864-40e6-904e-1bff72ec8b04"}]: dispatch 2024-06-14T22:33:52.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:52 smithi178 ceph-mon[32942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "33945598-f864-40e6-904e-1bff72ec8b04"}]': finished 2024-06-14T22:33:52.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:52 smithi178 ceph-mon[32942]: osdmap e5: 1 total, 0 up, 1 in 2024-06-14T22:33:52.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:52 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-14T22:33:52.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:52 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/3236510873' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:33:53.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:53 smithi046 ceph-mon[29586]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:53.442 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:53 smithi046 ceph-mon[38286]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:53.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:53 smithi178 ceph-mon[32942]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:55.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:55 smithi046 ceph-mon[29586]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:55.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:55 smithi046 ceph-mon[38286]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:55.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:55 smithi178 ceph-mon[32942]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:57.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:57 smithi046 ceph-mon[29586]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:57.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:57 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-14T22:33:57.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:57 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:57.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:57 smithi046 ceph-mon[38286]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:57.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:57 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-14T22:33:57.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:57 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:57.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:57 smithi178 ceph-mon[32942]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:57.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:57 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-14T22:33:57.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:57 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:33:58.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:58 smithi046 ceph-mon[38286]: Deploying daemon osd.0 on smithi046 2024-06-14T22:33:58.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:58 smithi046 ceph-mon[29586]: Deploying daemon osd.0 on smithi046 2024-06-14T22:33:58.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:58 smithi178 ceph-mon[32942]: Deploying daemon osd.0 on smithi046 2024-06-14T22:33:59.665 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:33:59 smithi046 ceph-mon[29586]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:59.665 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:33:59 smithi046 ceph-mon[38286]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:33:59.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:33:59 smithi178 ceph-mon[32942]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:34:01.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:01 smithi046 ceph-mon[29586]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:34:01.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:01 smithi046 ceph-mon[38286]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:34:01.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:01 smithi178 ceph-mon[32942]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:34:02.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:02 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:34:02.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:02 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:34:02.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:02 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:34:03.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:03 smithi046 ceph-mon[38286]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:34:03.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:03 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:03.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:03 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:03.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:03 smithi046 ceph-mon[29586]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:34:03.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:03 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:03.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:03 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:03.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:03 smithi178 ceph-mon[32942]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:34:03.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:03 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:03.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:03 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:05.524 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:05 smithi046 ceph-mon[29586]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:34:05.524 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:05 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:05.524 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:05 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:05.524 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:05 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:05.524 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:05 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:34:05.524 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:05 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:05.524 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:05 smithi046 ceph-mon[38286]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:34:05.525 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:05 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:05.525 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:05 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:05.525 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:05 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:05.525 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:05 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:34:05.525 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:05 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:05.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:05 smithi178 ceph-mon[32942]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:34:05.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:05 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:05.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:05 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:05.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:05 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:05.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:05 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:34:05.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:05 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:05.860 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 0 on host 'smithi046' 2024-06-14T22:34:06.486 DEBUG:teuthology.orchestra.run.smithi046:osd.0> sudo journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.0.service 2024-06-14T22:34:06.489 INFO:tasks.cephadm:Deploying osd.1 on smithi046 with /dev/vg_nvme/lv_3... 2024-06-14T22:34:06.489 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-14T22:34:06.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:06 smithi046 ceph-mon[29586]: from='osd.0 [v2:172.21.15.46:6802/2467893990,v1:172.21.15.46:6803/2467893990]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-14T22:34:06.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:06 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:34:06.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:06 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:06.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:06 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:06.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:06 smithi046 ceph-mon[38286]: from='osd.0 [v2:172.21.15.46:6802/2467893990,v1:172.21.15.46:6803/2467893990]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-14T22:34:06.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:06 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:34:06.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:06 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:06.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:06 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:06.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:06 smithi178 ceph-mon[32942]: from='osd.0 [v2:172.21.15.46:6802/2467893990,v1:172.21.15.46:6803/2467893990]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-14T22:34:06.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:06 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:34:06.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:06 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:06.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:06 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:07.275 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:34:07.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:07 smithi046 ceph-mon[38286]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:34:07.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:07 smithi046 ceph-mon[38286]: from='osd.0 [v2:172.21.15.46:6802/2467893990,v1:172.21.15.46:6803/2467893990]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-14T22:34:07.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:07 smithi046 ceph-mon[38286]: osdmap e6: 1 total, 0 up, 1 in 2024-06-14T22:34:07.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:07 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-14T22:34:07.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:07 smithi046 ceph-mon[38286]: from='osd.0 [v2:172.21.15.46:6802/2467893990,v1:172.21.15.46:6803/2467893990]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-06-14T22:34:07.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:07 smithi046 ceph-mon[29586]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:34:07.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:07 smithi046 ceph-mon[29586]: from='osd.0 [v2:172.21.15.46:6802/2467893990,v1:172.21.15.46:6803/2467893990]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-14T22:34:07.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:07 smithi046 ceph-mon[29586]: osdmap e6: 1 total, 0 up, 1 in 2024-06-14T22:34:07.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:07 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-14T22:34:07.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:07 smithi046 ceph-mon[29586]: from='osd.0 [v2:172.21.15.46:6802/2467893990,v1:172.21.15.46:6803/2467893990]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-06-14T22:34:07.693 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:34:07 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-0[42679]: 2024-06-14T22:34:07.436+0000 7fc70419d640 -1 osd.0 0 waiting for initial osdmap 2024-06-14T22:34:07.694 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:34:07 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-0[42679]: 2024-06-14T22:34:07.438+0000 7fc6fefd4640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-14T22:34:07.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:07 smithi178 ceph-mon[32942]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:34:07.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:07 smithi178 ceph-mon[32942]: from='osd.0 [v2:172.21.15.46:6802/2467893990,v1:172.21.15.46:6803/2467893990]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-14T22:34:07.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:07 smithi178 ceph-mon[32942]: osdmap e6: 1 total, 0 up, 1 in 2024-06-14T22:34:07.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:07 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-14T22:34:07.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:07 smithi178 ceph-mon[32942]: from='osd.0 [v2:172.21.15.46:6802/2467893990,v1:172.21.15.46:6803/2467893990]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-06-14T22:34:08.691 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:08 smithi046 ceph-mon[29586]: from='osd.0 [v2:172.21.15.46:6802/2467893990,v1:172.21.15.46:6803/2467893990]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-06-14T22:34:08.691 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:08 smithi046 ceph-mon[29586]: osdmap e7: 1 total, 0 up, 1 in 2024-06-14T22:34:08.691 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:08 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-14T22:34:08.691 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:08 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-14T22:34:08.691 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:08 smithi046 ceph-mon[38286]: from='osd.0 [v2:172.21.15.46:6802/2467893990,v1:172.21.15.46:6803/2467893990]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-06-14T22:34:08.691 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:08 smithi046 ceph-mon[38286]: osdmap e7: 1 total, 0 up, 1 in 2024-06-14T22:34:08.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:08 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-14T22:34:08.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:08 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-14T22:34:08.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:08 smithi178 ceph-mon[32942]: from='osd.0 [v2:172.21.15.46:6802/2467893990,v1:172.21.15.46:6803/2467893990]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-06-14T22:34:08.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:08 smithi178 ceph-mon[32942]: osdmap e7: 1 total, 0 up, 1 in 2024-06-14T22:34:08.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:08 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-14T22:34:08.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:08 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-14T22:34:09.487 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:09 smithi046 ceph-mon[29586]: purged_snaps scrub starts 2024-06-14T22:34:09.488 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:09 smithi046 ceph-mon[29586]: purged_snaps scrub ok 2024-06-14T22:34:09.488 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:09 smithi046 ceph-mon[29586]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:34:09.488 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:09 smithi046 ceph-mon[29586]: osd.0 [v2:172.21.15.46:6802/2467893990,v1:172.21.15.46:6803/2467893990] boot 2024-06-14T22:34:09.488 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:09 smithi046 ceph-mon[29586]: osdmap e8: 1 total, 1 up, 1 in 2024-06-14T22:34:09.488 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:09 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-14T22:34:09.488 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:09 smithi046 ceph-mon[38286]: purged_snaps scrub starts 2024-06-14T22:34:09.488 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:09 smithi046 ceph-mon[38286]: purged_snaps scrub ok 2024-06-14T22:34:09.488 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:09 smithi046 ceph-mon[38286]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:34:09.488 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:09 smithi046 ceph-mon[38286]: osd.0 [v2:172.21.15.46:6802/2467893990,v1:172.21.15.46:6803/2467893990] boot 2024-06-14T22:34:09.489 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:09 smithi046 ceph-mon[38286]: osdmap e8: 1 total, 1 up, 1 in 2024-06-14T22:34:09.489 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:09 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-14T22:34:09.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:09 smithi178 ceph-mon[32942]: purged_snaps scrub starts 2024-06-14T22:34:09.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:09 smithi178 ceph-mon[32942]: purged_snaps scrub ok 2024-06-14T22:34:09.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:09 smithi178 ceph-mon[32942]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-14T22:34:09.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:09 smithi178 ceph-mon[32942]: osd.0 [v2:172.21.15.46:6802/2467893990,v1:172.21.15.46:6803/2467893990] boot 2024-06-14T22:34:09.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:09 smithi178 ceph-mon[32942]: osdmap e8: 1 total, 1 up, 1 in 2024-06-14T22:34:09.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:09 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-14T22:34:10.480 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-14T22:34:10.564 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-14T22:34:10.574 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-06-14T22:34:10.575 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-06-14T22:34:10.575 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00835458 s, 1.3 GB/s 2024-06-14T22:34:10.575 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-06-14T22:34:10.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:10 smithi046 ceph-mon[38286]: osdmap e9: 1 total, 1 up, 1 in 2024-06-14T22:34:10.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:10 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:10.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:10 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:10.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:10 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:34:10.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:10 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:10.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:10 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:10.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:10 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:34:10.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:10 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:10.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:10 smithi046 ceph-mon[29586]: osdmap e9: 1 total, 1 up, 1 in 2024-06-14T22:34:10.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:10 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:10.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:10 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:10.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:10 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:34:10.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:10 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:10.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:10 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:10.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:10 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:34:10.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:10 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:10.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:10 smithi178 ceph-mon[32942]: osdmap e9: 1 total, 1 up, 1 in 2024-06-14T22:34:10.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:10 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:10.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:10 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:10.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:10 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:34:10.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:10 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:10.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:10 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:10.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:10 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:34:10.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:10 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:11.162 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_3 2024-06-14T22:34:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:34:11.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:11 smithi046 ceph-mon[29586]: Detected new or changed devices on smithi046 2024-06-14T22:34:11.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:11 smithi046 ceph-mon[29586]: Adjusting osd_memory_target on smithi046 to 16068M 2024-06-14T22:34:11.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:11 smithi046 ceph-mon[29586]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:11.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:11 smithi046 ceph-mon[38286]: Detected new or changed devices on smithi046 2024-06-14T22:34:11.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:11 smithi046 ceph-mon[38286]: Adjusting osd_memory_target on smithi046 to 16068M 2024-06-14T22:34:11.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:11 smithi046 ceph-mon[38286]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:11.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:11 smithi178 ceph-mon[32942]: Detected new or changed devices on smithi046 2024-06-14T22:34:11.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:11 smithi178 ceph-mon[32942]: Adjusting osd_memory_target on smithi046 to 16068M 2024-06-14T22:34:11.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:11 smithi178 ceph-mon[32942]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:13.715 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:13 smithi046 ceph-mon[29586]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:13.715 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:13 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:34:13.715 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:13 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:34:13.716 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:13 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:13.716 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:13 smithi046 ceph-mon[38286]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:13.716 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:13 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:34:13.716 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:13 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:34:13.716 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:13 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:13.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:13 smithi178 ceph-mon[32942]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:13.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:13 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:34:13.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:13 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:34:13.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:13 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:14.543 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:14 smithi046 ceph-mon[38286]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:34:14.543 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:14 smithi046 ceph-mon[29586]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:34:14.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:14 smithi178 ceph-mon[32942]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:34:15.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:15 smithi046 ceph-mon[38286]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:15.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:15 smithi046 ceph-mon[29586]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:15.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:15 smithi178 ceph-mon[32942]: pgmap v44: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:16.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:16 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/4185061431' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48f0e5ef-2d0a-4858-a629-69f82c604f9b"}]: dispatch 2024-06-14T22:34:16.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:16 smithi178 ceph-mon[32942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48f0e5ef-2d0a-4858-a629-69f82c604f9b"}]: dispatch 2024-06-14T22:34:16.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:16 smithi178 ceph-mon[32942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48f0e5ef-2d0a-4858-a629-69f82c604f9b"}]': finished 2024-06-14T22:34:16.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:16 smithi178 ceph-mon[32942]: osdmap e10: 2 total, 1 up, 2 in 2024-06-14T22:34:16.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:16 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-14T22:34:16.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:16 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/1159268675' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:34:16.942 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:16 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/4185061431' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48f0e5ef-2d0a-4858-a629-69f82c604f9b"}]: dispatch 2024-06-14T22:34:16.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:16 smithi046 ceph-mon[38286]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48f0e5ef-2d0a-4858-a629-69f82c604f9b"}]: dispatch 2024-06-14T22:34:16.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:16 smithi046 ceph-mon[38286]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48f0e5ef-2d0a-4858-a629-69f82c604f9b"}]': finished 2024-06-14T22:34:16.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:16 smithi046 ceph-mon[38286]: osdmap e10: 2 total, 1 up, 2 in 2024-06-14T22:34:16.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:16 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-14T22:34:16.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:16 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/1159268675' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:34:16.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:16 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/4185061431' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48f0e5ef-2d0a-4858-a629-69f82c604f9b"}]: dispatch 2024-06-14T22:34:16.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:16 smithi046 ceph-mon[29586]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48f0e5ef-2d0a-4858-a629-69f82c604f9b"}]: dispatch 2024-06-14T22:34:16.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:16 smithi046 ceph-mon[29586]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48f0e5ef-2d0a-4858-a629-69f82c604f9b"}]': finished 2024-06-14T22:34:16.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:16 smithi046 ceph-mon[29586]: osdmap e10: 2 total, 1 up, 2 in 2024-06-14T22:34:16.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:16 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-14T22:34:16.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:16 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/1159268675' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:34:17.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:17 smithi178 ceph-mon[32942]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:17.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:17 smithi046 ceph-mon[29586]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:17.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:17 smithi046 ceph-mon[38286]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:19.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:19 smithi046 ceph-mon[38286]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:19.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:19 smithi046 ceph-mon[29586]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:19.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:19 smithi178 ceph-mon[32942]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:21.677 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:21 smithi046 ceph-mon[29586]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:21.677 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:21 smithi046 ceph-mon[38286]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:22.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:21 smithi178 ceph-mon[32942]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:22.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:22 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-14T22:34:22.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:22 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:22.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:22 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-14T22:34:22.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:22 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:23.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:22 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-14T22:34:23.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:22 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:23.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:23 smithi046 ceph-mon[29586]: Deploying daemon osd.1 on smithi046 2024-06-14T22:34:23.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:23 smithi046 ceph-mon[29586]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:23.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:23 smithi046 ceph-mon[38286]: Deploying daemon osd.1 on smithi046 2024-06-14T22:34:23.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:23 smithi046 ceph-mon[38286]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:24.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:23 smithi178 ceph-mon[32942]: Deploying daemon osd.1 on smithi046 2024-06-14T22:34:24.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:23 smithi178 ceph-mon[32942]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:25.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:25 smithi046 ceph-mon[29586]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:25.660 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:25 smithi046 ceph-mon[38286]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:25.826 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:25 smithi178 ceph-mon[32942]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:27.656 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:27 smithi046 ceph-mon[29586]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:27.656 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:27 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:34:27.656 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:27 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:27.656 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:27 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:27.657 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:27 smithi046 ceph-mon[38286]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:27.657 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:34:27.657 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:27.657 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:28.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:27 smithi178 ceph-mon[32942]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:28.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:27 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:34:28.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:27 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:28.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:27 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:29.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:29 smithi046 ceph-mon[29586]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:29.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:29 smithi046 ceph-mon[38286]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:30.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:29 smithi178 ceph-mon[32942]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:30.382 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 1 on host 'smithi046' 2024-06-14T22:34:30.648 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:30.942 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:30.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:30.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:30.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:34:30.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:30.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[38286]: from='osd.1 [v2:172.21.15.46:6810/2133775041,v1:172.21.15.46:6811/2133775041]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-14T22:34:30.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[38286]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-14T22:34:30.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:34:30.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:30.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:30.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:30.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:30.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:34:30.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:30.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[29586]: from='osd.1 [v2:172.21.15.46:6810/2133775041,v1:172.21.15.46:6811/2133775041]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-14T22:34:30.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[29586]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-14T22:34:30.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:34:30.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:30.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:30 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:31.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:30 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:31.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:30 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:31.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:30 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:31.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:30 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:34:31.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:30 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:31.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:30 smithi178 ceph-mon[32942]: from='osd.1 [v2:172.21.15.46:6810/2133775041,v1:172.21.15.46:6811/2133775041]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-14T22:34:31.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:30 smithi178 ceph-mon[32942]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-14T22:34:31.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:30 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:34:31.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:30 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:31.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:30 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:31.062 DEBUG:teuthology.orchestra.run.smithi046:osd.1> sudo journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.1.service 2024-06-14T22:34:31.064 INFO:tasks.cephadm:Deploying osd.2 on smithi046 with /dev/vg_nvme/lv_2... 2024-06-14T22:34:31.065 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-14T22:34:31.788 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:34:31.894 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:31 smithi046 ceph-mon[29586]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:31.895 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:31 smithi046 ceph-mon[29586]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-14T22:34:31.895 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:31 smithi046 ceph-mon[29586]: osdmap e11: 2 total, 1 up, 2 in 2024-06-14T22:34:31.895 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:31 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-14T22:34:31.895 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:31 smithi046 ceph-mon[29586]: from='osd.1 [v2:172.21.15.46:6810/2133775041,v1:172.21.15.46:6811/2133775041]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-06-14T22:34:31.895 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:31 smithi046 ceph-mon[29586]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-06-14T22:34:31.895 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:31 smithi046 ceph-mon[38286]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:31.895 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:31 smithi046 ceph-mon[38286]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-14T22:34:31.896 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:31 smithi046 ceph-mon[38286]: osdmap e11: 2 total, 1 up, 2 in 2024-06-14T22:34:31.896 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:31 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-14T22:34:31.896 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:31 smithi046 ceph-mon[38286]: from='osd.1 [v2:172.21.15.46:6810/2133775041,v1:172.21.15.46:6811/2133775041]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-06-14T22:34:31.896 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:31 smithi046 ceph-mon[38286]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-06-14T22:34:31.896 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:34:31 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-1[48593]: 2024-06-14T22:34:31.672+0000 7f4b5eb39640 -1 osd.1 0 waiting for initial osdmap 2024-06-14T22:34:31.896 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:34:31 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-1[48593]: 2024-06-14T22:34:31.675+0000 7f4b5916f640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-14T22:34:32.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:31 smithi178 ceph-mon[32942]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:32.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:31 smithi178 ceph-mon[32942]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-14T22:34:32.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:31 smithi178 ceph-mon[32942]: osdmap e11: 2 total, 1 up, 2 in 2024-06-14T22:34:32.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:31 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-14T22:34:32.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:31 smithi178 ceph-mon[32942]: from='osd.1 [v2:172.21.15.46:6810/2133775041,v1:172.21.15.46:6811/2133775041]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-06-14T22:34:32.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:31 smithi178 ceph-mon[32942]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-06-14T22:34:32.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:32 smithi046 ceph-mon[29586]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-06-14T22:34:32.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:32 smithi046 ceph-mon[29586]: osdmap e12: 2 total, 1 up, 2 in 2024-06-14T22:34:32.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:32 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-14T22:34:32.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:32 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-14T22:34:32.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:32 smithi046 ceph-mon[38286]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-06-14T22:34:32.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:32 smithi046 ceph-mon[38286]: osdmap e12: 2 total, 1 up, 2 in 2024-06-14T22:34:32.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:32 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-14T22:34:32.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:32 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-14T22:34:33.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:32 smithi178 ceph-mon[32942]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-06-14T22:34:33.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:32 smithi178 ceph-mon[32942]: osdmap e12: 2 total, 1 up, 2 in 2024-06-14T22:34:33.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:32 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-14T22:34:33.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:32 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-14T22:34:33.923 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:33 smithi046 ceph-mon[29586]: purged_snaps scrub starts 2024-06-14T22:34:33.924 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:33 smithi046 ceph-mon[29586]: purged_snaps scrub ok 2024-06-14T22:34:33.924 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:33 smithi046 ceph-mon[29586]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:33.924 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:33 smithi046 ceph-mon[29586]: osd.1 [v2:172.21.15.46:6810/2133775041,v1:172.21.15.46:6811/2133775041] boot 2024-06-14T22:34:33.924 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:33 smithi046 ceph-mon[29586]: osdmap e13: 2 total, 2 up, 2 in 2024-06-14T22:34:33.924 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:33 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-14T22:34:33.924 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:33 smithi046 ceph-mon[38286]: purged_snaps scrub starts 2024-06-14T22:34:33.924 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:33 smithi046 ceph-mon[38286]: purged_snaps scrub ok 2024-06-14T22:34:33.924 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:33 smithi046 ceph-mon[38286]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:33.924 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:33 smithi046 ceph-mon[38286]: osd.1 [v2:172.21.15.46:6810/2133775041,v1:172.21.15.46:6811/2133775041] boot 2024-06-14T22:34:33.924 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:33 smithi046 ceph-mon[38286]: osdmap e13: 2 total, 2 up, 2 in 2024-06-14T22:34:33.925 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:33 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-14T22:34:34.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:33 smithi178 ceph-mon[32942]: purged_snaps scrub starts 2024-06-14T22:34:34.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:33 smithi178 ceph-mon[32942]: purged_snaps scrub ok 2024-06-14T22:34:34.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:33 smithi178 ceph-mon[32942]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-06-14T22:34:34.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:33 smithi178 ceph-mon[32942]: osd.1 [v2:172.21.15.46:6810/2133775041,v1:172.21.15.46:6811/2133775041] boot 2024-06-14T22:34:34.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:33 smithi178 ceph-mon[32942]: osdmap e13: 2 total, 2 up, 2 in 2024-06-14T22:34:34.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:33 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-14T22:34:34.905 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-14T22:34:34.905 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[38286]: osdmap e14: 2 total, 2 up, 2 in 2024-06-14T22:34:34.905 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[38286]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:34.905 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[38286]: Detected new or changed devices on smithi046 2024-06-14T22:34:34.905 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:34.905 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:34.905 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:34:34.905 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:34:34.906 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[38286]: Adjusting osd_memory_target on smithi046 to 8034M 2024-06-14T22:34:34.906 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:34.906 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:34.906 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:34:34.906 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:34.906 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[29586]: osdmap e14: 2 total, 2 up, 2 in 2024-06-14T22:34:34.906 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[29586]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:34.906 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[29586]: Detected new or changed devices on smithi046 2024-06-14T22:34:34.906 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:34.906 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:34.907 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:34:34.907 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:34:34.907 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[29586]: Adjusting osd_memory_target on smithi046 to 8034M 2024-06-14T22:34:34.907 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:34.907 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:34.907 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:34:34.907 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:34 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:34.997 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-14T22:34:35.008 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-06-14T22:34:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-06-14T22:34:35.008 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00835818 s, 1.3 GB/s 2024-06-14T22:34:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-06-14T22:34:35.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:34 smithi178 ceph-mon[32942]: osdmap e14: 2 total, 2 up, 2 in 2024-06-14T22:34:35.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:34 smithi178 ceph-mon[32942]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:35.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:34 smithi178 ceph-mon[32942]: Detected new or changed devices on smithi046 2024-06-14T22:34:35.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:34 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:35.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:34 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:35.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:34 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:34:35.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:34 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:34:35.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:34 smithi178 ceph-mon[32942]: Adjusting osd_memory_target on smithi046 to 8034M 2024-06-14T22:34:35.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:34 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:35.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:34 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:35.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:34 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:34:35.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:34 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:35.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_2 2024-06-14T22:34:35.858 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:34:37.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:37 smithi046 ceph-mon[29586]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:37.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:37 smithi046 ceph-mon[38286]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:37.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:37 smithi178 ceph-mon[32942]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:38.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:38 smithi046 ceph-mon[29586]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:34:38.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:38 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:34:38.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:38 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:34:38.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:38 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:38.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:38 smithi046 ceph-mon[38286]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:34:38.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:38 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:34:38.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:38 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:34:38.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:38 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:38.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:38 smithi178 ceph-mon[32942]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:34:38.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:38 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:34:38.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:38 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:34:38.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:38 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:39.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:39 smithi046 ceph-mon[29586]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:39.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:39 smithi046 ceph-mon[38286]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:39.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:39 smithi178 ceph-mon[32942]: pgmap v61: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:40.604 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:40 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/2439776464' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74b998f5-fb99-4c37-9463-11dd602f1364"}]: dispatch 2024-06-14T22:34:40.604 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:40 smithi046 ceph-mon[38286]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74b998f5-fb99-4c37-9463-11dd602f1364"}]: dispatch 2024-06-14T22:34:40.604 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:40 smithi046 ceph-mon[38286]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74b998f5-fb99-4c37-9463-11dd602f1364"}]': finished 2024-06-14T22:34:40.604 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:40 smithi046 ceph-mon[38286]: osdmap e15: 3 total, 2 up, 3 in 2024-06-14T22:34:40.604 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:40 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-14T22:34:40.605 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:40 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/2439776464' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74b998f5-fb99-4c37-9463-11dd602f1364"}]: dispatch 2024-06-14T22:34:40.605 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:40 smithi046 ceph-mon[29586]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74b998f5-fb99-4c37-9463-11dd602f1364"}]: dispatch 2024-06-14T22:34:40.605 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:40 smithi046 ceph-mon[29586]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74b998f5-fb99-4c37-9463-11dd602f1364"}]': finished 2024-06-14T22:34:40.605 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:40 smithi046 ceph-mon[29586]: osdmap e15: 3 total, 2 up, 3 in 2024-06-14T22:34:40.605 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:40 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-14T22:34:40.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:40 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/2439776464' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74b998f5-fb99-4c37-9463-11dd602f1364"}]: dispatch 2024-06-14T22:34:40.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:40 smithi178 ceph-mon[32942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "74b998f5-fb99-4c37-9463-11dd602f1364"}]: dispatch 2024-06-14T22:34:40.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:40 smithi178 ceph-mon[32942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "74b998f5-fb99-4c37-9463-11dd602f1364"}]': finished 2024-06-14T22:34:40.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:40 smithi178 ceph-mon[32942]: osdmap e15: 3 total, 2 up, 3 in 2024-06-14T22:34:40.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:40 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-14T22:34:41.470 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:41 smithi046 ceph-mon[29586]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:41.470 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:41 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/3868471614' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:34:41.471 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:41 smithi046 ceph-mon[38286]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:41.471 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:41 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/3868471614' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:34:41.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:41 smithi178 ceph-mon[32942]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:41.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:41 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/3868471614' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:34:43.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:43 smithi046 ceph-mon[29586]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:43.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:43 smithi046 ceph-mon[38286]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:43.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:43 smithi178 ceph-mon[32942]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:45.586 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:45 smithi046 ceph-mon[29586]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:45.586 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:45 smithi046 ceph-mon[38286]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:45.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:45 smithi178 ceph-mon[32942]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:47.574 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:47 smithi046 ceph-mon[29586]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:47.574 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:47 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-14T22:34:47.574 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:47 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:47.574 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:47 smithi046 ceph-mon[29586]: Deploying daemon osd.2 on smithi046 2024-06-14T22:34:47.575 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:47 smithi046 ceph-mon[38286]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:47.575 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:47 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-14T22:34:47.575 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:47 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:47.575 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:47 smithi046 ceph-mon[38286]: Deploying daemon osd.2 on smithi046 2024-06-14T22:34:47.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:47 smithi178 ceph-mon[32942]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:47.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:47 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-14T22:34:47.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:47 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:47.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:47 smithi178 ceph-mon[32942]: Deploying daemon osd.2 on smithi046 2024-06-14T22:34:49.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:49 smithi046 ceph-mon[29586]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:49.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:49 smithi046 ceph-mon[38286]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:49.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:49 smithi178 ceph-mon[32942]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:51.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:51 smithi046 ceph-mon[29586]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:51.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:51 smithi046 ceph-mon[38286]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:51.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:51 smithi178 ceph-mon[32942]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:52.519 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:52 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:34:52.519 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:52 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:52.519 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:52 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:52.520 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:52 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:34:52.520 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:52 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:52.520 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:52 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:52.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:52 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:34:52.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:52 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:52.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:52 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:53.488 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:53 smithi046 ceph-mon[29586]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:53.489 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:53 smithi046 ceph-mon[38286]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:53.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:53 smithi178 ceph-mon[32942]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:54.993 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 2 on host 'smithi046' 2024-06-14T22:34:55.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:55 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:55.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:55 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:55.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:55 smithi178 ceph-mon[32942]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:55.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:55 smithi178 ceph-mon[32942]: from='osd.2 [v2:172.21.15.46:6818/3036855066,v1:172.21.15.46:6819/3036855066]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-14T22:34:55.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:55 smithi178 ceph-mon[32942]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-14T22:34:55.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:55 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:55.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:55 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:34:55.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:55 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:55.526 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:55 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:34:55.526 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:55 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:55.526 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:55 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:55.649 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:55.650 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:55.650 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[38286]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:55.650 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[38286]: from='osd.2 [v2:172.21.15.46:6818/3036855066,v1:172.21.15.46:6819/3036855066]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-14T22:34:55.650 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[38286]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-14T22:34:55.650 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:55.650 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:34:55.650 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:55.650 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:34:55.650 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:55.650 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:55.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:55.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:55.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[29586]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:55.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[29586]: from='osd.2 [v2:172.21.15.46:6818/3036855066,v1:172.21.15.46:6819/3036855066]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-14T22:34:55.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[29586]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-14T22:34:55.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:34:55.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:34:55.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:55.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:34:55.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:55.651 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:55 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:34:55.679 DEBUG:teuthology.orchestra.run.smithi046:osd.2> sudo journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.2.service 2024-06-14T22:34:55.682 INFO:tasks.cephadm:Deploying osd.3 on smithi046 with /dev/vg_nvme/lv_1... 2024-06-14T22:34:55.682 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-14T22:34:56.442 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:56 smithi046 ceph-mon[38286]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-14T22:34:56.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:56 smithi046 ceph-mon[38286]: osdmap e16: 3 total, 2 up, 3 in 2024-06-14T22:34:56.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:56 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-14T22:34:56.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:56 smithi046 ceph-mon[38286]: from='osd.2 [v2:172.21.15.46:6818/3036855066,v1:172.21.15.46:6819/3036855066]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-06-14T22:34:56.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:56 smithi046 ceph-mon[38286]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-06-14T22:34:56.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:56 smithi046 ceph-mon[29586]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-14T22:34:56.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:56 smithi046 ceph-mon[29586]: osdmap e16: 3 total, 2 up, 3 in 2024-06-14T22:34:56.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:56 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-14T22:34:56.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:56 smithi046 ceph-mon[29586]: from='osd.2 [v2:172.21.15.46:6818/3036855066,v1:172.21.15.46:6819/3036855066]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-06-14T22:34:56.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:56 smithi046 ceph-mon[29586]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-06-14T22:34:56.444 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:34:56 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-2[54842]: 2024-06-14T22:34:56.245+0000 7f3c5af9a640 -1 osd.2 0 waiting for initial osdmap 2024-06-14T22:34:56.444 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:34:56 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-2[54842]: 2024-06-14T22:34:56.248+0000 7f3c555d0640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-14T22:34:56.482 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:34:56.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:56 smithi178 ceph-mon[32942]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-14T22:34:56.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:56 smithi178 ceph-mon[32942]: osdmap e16: 3 total, 2 up, 3 in 2024-06-14T22:34:56.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:56 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-14T22:34:56.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:56 smithi178 ceph-mon[32942]: from='osd.2 [v2:172.21.15.46:6818/3036855066,v1:172.21.15.46:6819/3036855066]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-06-14T22:34:56.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:56 smithi178 ceph-mon[32942]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-06-14T22:34:57.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:57 smithi178 ceph-mon[32942]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-06-14T22:34:57.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:57 smithi178 ceph-mon[32942]: osdmap e17: 3 total, 2 up, 3 in 2024-06-14T22:34:57.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:57 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-14T22:34:57.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:57 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-14T22:34:57.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:57 smithi178 ceph-mon[32942]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:57.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:57 smithi046 ceph-mon[38286]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-06-14T22:34:57.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:57 smithi046 ceph-mon[38286]: osdmap e17: 3 total, 2 up, 3 in 2024-06-14T22:34:57.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:57 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-14T22:34:57.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:57 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-14T22:34:57.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:57 smithi046 ceph-mon[38286]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:57.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:57 smithi046 ceph-mon[29586]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-06-14T22:34:57.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:57 smithi046 ceph-mon[29586]: osdmap e17: 3 total, 2 up, 3 in 2024-06-14T22:34:57.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:57 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-14T22:34:57.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:57 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-14T22:34:57.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:57 smithi046 ceph-mon[29586]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-06-14T22:34:58.417 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:58 smithi046 ceph-mon[38286]: purged_snaps scrub starts 2024-06-14T22:34:58.417 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:58 smithi046 ceph-mon[38286]: purged_snaps scrub ok 2024-06-14T22:34:58.417 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:58 smithi046 ceph-mon[38286]: osd.2 [v2:172.21.15.46:6818/3036855066,v1:172.21.15.46:6819/3036855066] boot 2024-06-14T22:34:58.417 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:58 smithi046 ceph-mon[38286]: osdmap e18: 3 total, 3 up, 3 in 2024-06-14T22:34:58.417 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:58 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-14T22:34:58.418 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:58 smithi046 ceph-mon[29586]: purged_snaps scrub starts 2024-06-14T22:34:58.418 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:58 smithi046 ceph-mon[29586]: purged_snaps scrub ok 2024-06-14T22:34:58.418 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:58 smithi046 ceph-mon[29586]: osd.2 [v2:172.21.15.46:6818/3036855066,v1:172.21.15.46:6819/3036855066] boot 2024-06-14T22:34:58.418 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:58 smithi046 ceph-mon[29586]: osdmap e18: 3 total, 3 up, 3 in 2024-06-14T22:34:58.418 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:58 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-14T22:34:58.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:58 smithi178 ceph-mon[32942]: purged_snaps scrub starts 2024-06-14T22:34:58.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:58 smithi178 ceph-mon[32942]: purged_snaps scrub ok 2024-06-14T22:34:58.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:58 smithi178 ceph-mon[32942]: osd.2 [v2:172.21.15.46:6818/3036855066,v1:172.21.15.46:6819/3036855066] boot 2024-06-14T22:34:58.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:58 smithi178 ceph-mon[32942]: osdmap e18: 3 total, 3 up, 3 in 2024-06-14T22:34:58.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:58 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-14T22:34:59.112 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-14T22:34:59.209 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-14T22:34:59.219 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-06-14T22:34:59.220 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-06-14T22:34:59.220 INFO:teuthology.orchestra.run.smithi046.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00933284 s, 1.1 GB/s 2024-06-14T22:34:59.220 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-06-14T22:34:59.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:59 smithi046 ceph-mon[29586]: osdmap e19: 3 total, 3 up, 3 in 2024-06-14T22:34:59.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:59 smithi046 ceph-mon[29586]: pgmap v76: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:34:59.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:34:59 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-06-14T22:34:59.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:59 smithi046 ceph-mon[38286]: osdmap e19: 3 total, 3 up, 3 in 2024-06-14T22:34:59.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:59 smithi046 ceph-mon[38286]: pgmap v76: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:34:59.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:34:59 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-06-14T22:34:59.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:59 smithi178 ceph-mon[32942]: osdmap e19: 3 total, 3 up, 3 in 2024-06-14T22:34:59.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:59 smithi178 ceph-mon[32942]: pgmap v76: 0 pgs: ; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:34:59.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:34:59 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-06-14T22:34:59.875 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_1 2024-06-14T22:35:00.349 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:35:00.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:00 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-06-14T22:35:00.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:00 smithi178 ceph-mon[32942]: osdmap e20: 3 total, 3 up, 3 in 2024-06-14T22:35:00.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:00 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-14T22:35:00.536 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:00 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-06-14T22:35:00.536 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:00 smithi046 ceph-mon[29586]: osdmap e20: 3 total, 3 up, 3 in 2024-06-14T22:35:00.536 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:00 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-14T22:35:00.536 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:00 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-06-14T22:35:00.536 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:00 smithi046 ceph-mon[38286]: osdmap e20: 3 total, 3 up, 3 in 2024-06-14T22:35:00.536 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:00 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-14T22:35:01.192 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:35:00 smithi046 sudo[58997]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-14T22:35:01.192 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:35:00 smithi046 sudo[58997]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-14T22:35:01.192 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:35:00 smithi046 sudo[58997]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-14T22:35:01.192 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:35:00 smithi046 sudo[58997]: pam_unix(sudo:session): session closed for user root 2024-06-14T22:35:01.508 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:35:01 smithi046 sudo[59019]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-14T22:35:01.508 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:35:01 smithi046 sudo[59019]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-14T22:35:01.508 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:35:01 smithi046 sudo[59019]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-14T22:35:01.508 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:35:01 smithi046 sudo[59019]: pam_unix(sudo:session): session closed for user root 2024-06-14T22:35:01.509 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-14T22:35:01.509 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[38286]: osdmap e21: 3 total, 3 up, 3 in 2024-06-14T22:35:01.509 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[38286]: Detected new or changed devices on smithi046 2024-06-14T22:35:01.509 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:01.509 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:01.509 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:01.509 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:01.509 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:01.509 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[38286]: Adjusting osd_memory_target on smithi046 to 5356M 2024-06-14T22:35:01.509 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:01.510 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:01.510 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:35:01.510 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:01.510 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[38286]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:35:01.510 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-14T22:35:01.510 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[29586]: osdmap e21: 3 total, 3 up, 3 in 2024-06-14T22:35:01.510 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[29586]: Detected new or changed devices on smithi046 2024-06-14T22:35:01.510 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:01.510 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:01.510 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:01.510 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:01.511 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:01.511 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[29586]: Adjusting osd_memory_target on smithi046 to 5356M 2024-06-14T22:35:01.511 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:01.511 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:01.511 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:35:01.511 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:01.511 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:01 smithi046 ceph-mon[29586]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:35:01.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:01 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-14T22:35:01.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:01 smithi178 ceph-mon[32942]: osdmap e21: 3 total, 3 up, 3 in 2024-06-14T22:35:01.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:01 smithi178 ceph-mon[32942]: Detected new or changed devices on smithi046 2024-06-14T22:35:01.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:01 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:01.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:01 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:01.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:01 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:01.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:01 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:01.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:01 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:01.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:01 smithi178 ceph-mon[32942]: Adjusting osd_memory_target on smithi046 to 5356M 2024-06-14T22:35:01.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:01 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:01.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:01 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:01.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:01 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:35:01.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:01 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:01.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:01 smithi178 ceph-mon[32942]: pgmap v79: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:35:01.774 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:35:01 smithi046 sudo[59042]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-14T22:35:01.774 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:35:01 smithi046 sudo[59042]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-14T22:35:01.775 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:35:01 smithi046 sudo[59042]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-14T22:35:01.775 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:35:01 smithi046 sudo[59042]: pam_unix(sudo:session): session closed for user root 2024-06-14T22:35:02.442 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:35:02 smithi046 sudo[59091]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-14T22:35:02.442 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:35:02 smithi046 sudo[59091]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-14T22:35:02.442 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:35:02 smithi046 sudo[59091]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-14T22:35:02.443 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:35:02 smithi046 sudo[59091]: pam_unix(sudo:session): session closed for user root 2024-06-14T22:35:02.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:02 smithi046 ceph-mon[29586]: osdmap e22: 3 total, 3 up, 3 in 2024-06-14T22:35:02.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:02 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:02.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:02 smithi046 ceph-mon[29586]: mgrmap e15: y(active, since 2m), standbys: x 2024-06-14T22:35:02.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:02 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:35:02.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:02 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:35:02.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:02 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:02.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:02 smithi046 ceph-mon[38286]: osdmap e22: 3 total, 3 up, 3 in 2024-06-14T22:35:02.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:02 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:02.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:02 smithi046 ceph-mon[38286]: mgrmap e15: y(active, since 2m), standbys: x 2024-06-14T22:35:02.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:02 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:35:02.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:02 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:35:02.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:02 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:02.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:02 smithi178 ceph-mon[32942]: osdmap e22: 3 total, 3 up, 3 in 2024-06-14T22:35:02.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:02 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:02.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:02 smithi178 ceph-mon[32942]: mgrmap e15: y(active, since 2m), standbys: x 2024-06-14T22:35:02.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:02 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:35:02.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:02 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:35:02.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:02 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:02.922 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:35:02 smithi046 sudo[59215]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-14T22:35:02.922 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:35:02 smithi046 sudo[59215]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-14T22:35:02.922 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:35:02 smithi046 sudo[59215]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-14T22:35:02.922 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:35:02 smithi046 sudo[59215]: pam_unix(sudo:session): session closed for user root 2024-06-14T22:35:03.193 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:35:03 smithi046 sudo[59255]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-14T22:35:03.193 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:35:03 smithi046 sudo[59255]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-14T22:35:03.193 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:35:03 smithi046 sudo[59255]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-14T22:35:03.193 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:35:03 smithi046 sudo[59255]: pam_unix(sudo:session): session closed for user root 2024-06-14T22:35:03.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:03 smithi046 sudo[59267]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-14T22:35:03.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:03 smithi046 sudo[59267]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-14T22:35:03.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:03 smithi046 sudo[59267]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-14T22:35:03.659 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:03 smithi046 sudo[59267]: pam_unix(sudo:session): session closed for user root 2024-06-14T22:35:04.260 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:04 smithi178 ceph-mon[32942]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:35:04.260 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:04 smithi178 ceph-mon[32942]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:35:04.260 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:04 smithi178 ceph-mon[32942]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-14T22:35:04.260 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:04 smithi178 sudo[35667]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-14T22:35:04.260 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:04 smithi178 sudo[35667]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-14T22:35:04.260 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:04 smithi178 sudo[35667]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-14T22:35:04.261 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:04 smithi178 sudo[35667]: pam_unix(sudo:session): session closed for user root 2024-06-14T22:35:04.417 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:04 smithi046 ceph-mon[38286]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:35:04.417 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:04 smithi046 ceph-mon[38286]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:35:04.417 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:04 smithi046 ceph-mon[38286]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-14T22:35:04.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:03 smithi046 sudo[59312]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-14T22:35:04.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:03 smithi046 sudo[59312]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-14T22:35:04.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:03 smithi046 sudo[59312]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-14T22:35:04.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:04 smithi046 sudo[59312]: pam_unix(sudo:session): session closed for user root 2024-06-14T22:35:04.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:04 smithi046 ceph-mon[29586]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:35:04.418 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:04 smithi046 ceph-mon[29586]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:35:04.418 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:04 smithi046 ceph-mon[29586]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-14T22:35:04.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:04 smithi178 sudo[35670]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-14T22:35:04.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:04 smithi178 sudo[35670]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-14T22:35:04.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:04 smithi178 sudo[35670]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-14T22:35:04.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:04 smithi178 sudo[35670]: pam_unix(sudo:session): session closed for user root 2024-06-14T22:35:05.417 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:04 smithi046 sudo[59362]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-14T22:35:05.417 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:04 smithi046 sudo[59362]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-14T22:35:05.417 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:04 smithi046 sudo[59362]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-14T22:35:05.417 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:04 smithi046 sudo[59362]: pam_unix(sudo:session): session closed for user root 2024-06-14T22:35:05.680 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[29586]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-14T22:35:05.680 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:35:05.680 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:35:05.680 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:35:05.680 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[29586]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-14T22:35:05.680 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:35:05.681 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:35:05.681 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:35:05.681 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[29586]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:35:05.681 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[29586]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-14T22:35:05.681 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:05.681 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:35:05.681 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:35:05.681 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:35:05.681 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/643196656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4bd323c-2885-47a0-a44c-275077fc4b06"}]: dispatch 2024-06-14T22:35:05.681 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:05 smithi046 sudo[59365]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-14T22:35:05.681 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:05 smithi046 sudo[59365]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-14T22:35:05.682 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:05 smithi046 sudo[59365]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-06-14T22:35:05.682 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:05 smithi046 sudo[59365]: pam_unix(sudo:session): session closed for user root 2024-06-14T22:35:05.682 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[38286]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-14T22:35:05.682 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:35:05.682 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:35:05.682 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:35:05.682 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[38286]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-14T22:35:05.682 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:35:05.682 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:35:05.682 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:35:05.682 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[38286]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:35:05.683 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[38286]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-14T22:35:05.683 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:05.683 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:35:05.683 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:35:05.683 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:35:05.683 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:05 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/643196656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4bd323c-2885-47a0-a44c-275077fc4b06"}]: dispatch 2024-06-14T22:35:05.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:05 smithi178 ceph-mon[32942]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-14T22:35:05.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:05 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:35:05.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:05 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:35:05.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:05 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:35:05.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:05 smithi178 ceph-mon[32942]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-14T22:35:05.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:05 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:35:05.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:05 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:35:05.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:05 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:35:05.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:05 smithi178 ceph-mon[32942]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 873 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:35:05.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:05 smithi178 ceph-mon[32942]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-14T22:35:05.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:05 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:05.777 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:05 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:35:05.777 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:05 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:35:05.777 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:05 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:35:05.777 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:05 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/643196656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e4bd323c-2885-47a0-a44c-275077fc4b06"}]: dispatch 2024-06-14T22:35:06.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:06 smithi046 ceph-mon[38286]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-14T22:35:06.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:06 smithi046 ceph-mon[38286]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-14T22:35:06.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:06 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/643196656' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4bd323c-2885-47a0-a44c-275077fc4b06"}]': finished 2024-06-14T22:35:06.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:06 smithi046 ceph-mon[38286]: osdmap e23: 4 total, 3 up, 4 in 2024-06-14T22:35:06.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:06 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-14T22:35:06.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:06 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/3991881401' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:35:06.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:06 smithi046 ceph-mon[29586]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-14T22:35:06.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:06 smithi046 ceph-mon[29586]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-14T22:35:06.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:06 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/643196656' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4bd323c-2885-47a0-a44c-275077fc4b06"}]': finished 2024-06-14T22:35:06.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:06 smithi046 ceph-mon[29586]: osdmap e23: 4 total, 3 up, 4 in 2024-06-14T22:35:06.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:06 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-14T22:35:06.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:06 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/3991881401' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:35:06.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:06 smithi178 ceph-mon[32942]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-14T22:35:06.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:06 smithi178 ceph-mon[32942]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-14T22:35:06.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:06 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/643196656' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e4bd323c-2885-47a0-a44c-275077fc4b06"}]': finished 2024-06-14T22:35:06.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:06 smithi178 ceph-mon[32942]: osdmap e23: 4 total, 3 up, 4 in 2024-06-14T22:35:06.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:06 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-14T22:35:06.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:06 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/3991881401' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:35:07.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:07 smithi046 ceph-mon[29586]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:35:07.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:07 smithi046 ceph-mon[38286]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:35:07.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:07 smithi178 ceph-mon[32942]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:35:09.688 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:09 smithi046 ceph-mon[29586]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-06-14T22:35:09.688 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:09 smithi046 ceph-mon[38286]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-06-14T22:35:09.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:09 smithi178 ceph-mon[32942]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-06-14T22:35:11.599 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:11 smithi046 ceph-mon[29586]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-06-14T22:35:11.600 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:11 smithi046 ceph-mon[38286]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-06-14T22:35:11.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:11 smithi178 ceph-mon[32942]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-06-14T22:35:12.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:12 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-14T22:35:12.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:12 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:12.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:12 smithi046 ceph-mon[29586]: Deploying daemon osd.3 on smithi046 2024-06-14T22:35:12.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:12 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-14T22:35:12.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:12 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:12.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:12 smithi046 ceph-mon[38286]: Deploying daemon osd.3 on smithi046 2024-06-14T22:35:12.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:12 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-14T22:35:12.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:12 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:12.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:12 smithi178 ceph-mon[32942]: Deploying daemon osd.3 on smithi046 2024-06-14T22:35:13.618 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:13 smithi046 ceph-mon[29586]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-06-14T22:35:13.619 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:13 smithi046 ceph-mon[38286]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-06-14T22:35:13.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:13 smithi178 ceph-mon[32942]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-06-14T22:35:15.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:15 smithi046 ceph-mon[29586]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-06-14T22:35:15.610 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:15 smithi046 ceph-mon[38286]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-06-14T22:35:15.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:15 smithi178 ceph-mon[32942]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-06-14T22:35:17.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:17 smithi178 ceph-mon[32942]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.6 KiB/s rd, 72 KiB/s wr, 5 op/s 2024-06-14T22:35:17.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:17 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:35:17.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:17 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:17.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:17 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:17.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:17 smithi046 ceph-mon[29586]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.6 KiB/s rd, 72 KiB/s wr, 5 op/s 2024-06-14T22:35:17.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:17 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:35:17.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:17 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:17.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:17 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:17.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:17 smithi046 ceph-mon[38286]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.6 KiB/s rd, 72 KiB/s wr, 5 op/s 2024-06-14T22:35:17.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:17 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:35:17.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:17 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:17.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:17 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:19.649 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:19 smithi046 ceph-mon[29586]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-06-14T22:35:19.649 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:19 smithi046 ceph-mon[38286]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-06-14T22:35:19.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:19 smithi178 ceph-mon[32942]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-06-14T22:35:20.240 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 3 on host 'smithi046' 2024-06-14T22:35:20.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:20 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:20.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:20 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:20.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:20 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:20.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:20 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:35:20.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:20 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:20.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:20 smithi178 ceph-mon[32942]: from='osd.3 [v2:172.21.15.46:6826/2597052465,v1:172.21.15.46:6827/2597052465]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-14T22:35:20.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:20 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:35:20.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:20 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:20.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:20 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:20.867 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:20 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:20.867 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:20 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:20.867 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:20 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:20.867 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:20 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:35:20.867 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:20 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:20.867 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:20 smithi046 ceph-mon[38286]: from='osd.3 [v2:172.21.15.46:6826/2597052465,v1:172.21.15.46:6827/2597052465]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-14T22:35:20.868 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:20 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:35:20.868 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:20 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:20.868 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:20 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:20.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:20 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:20.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:20 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:20.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:20 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:20.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:20 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:35:20.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:20 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:20.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:20 smithi046 ceph-mon[29586]: from='osd.3 [v2:172.21.15.46:6826/2597052465,v1:172.21.15.46:6827/2597052465]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-14T22:35:20.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:20 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:35:20.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:20 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:20.869 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:20 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:20.891 DEBUG:teuthology.orchestra.run.smithi046:osd.3> sudo journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.3.service 2024-06-14T22:35:20.894 INFO:tasks.cephadm:Deploying osd.4 on smithi178 with /dev/vg_nvme/lv_4... 2024-06-14T22:35:20.894 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-14T22:35:21.148 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.b/config 2024-06-14T22:35:21.521 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:21 smithi178 ceph-mon[32942]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:35:21.521 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:21 smithi178 ceph-mon[32942]: from='osd.3 [v2:172.21.15.46:6826/2597052465,v1:172.21.15.46:6827/2597052465]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-14T22:35:21.521 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:21 smithi178 ceph-mon[32942]: osdmap e24: 4 total, 3 up, 4 in 2024-06-14T22:35:21.521 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:21 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-14T22:35:21.521 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:21 smithi178 ceph-mon[32942]: from='osd.3 [v2:172.21.15.46:6826/2597052465,v1:172.21.15.46:6827/2597052465]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-06-14T22:35:21.807 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:21 smithi046 ceph-mon[38286]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:35:21.807 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:21 smithi046 ceph-mon[38286]: from='osd.3 [v2:172.21.15.46:6826/2597052465,v1:172.21.15.46:6827/2597052465]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-14T22:35:21.807 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:21 smithi046 ceph-mon[38286]: osdmap e24: 4 total, 3 up, 4 in 2024-06-14T22:35:21.807 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:21 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-14T22:35:21.807 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:21 smithi046 ceph-mon[38286]: from='osd.3 [v2:172.21.15.46:6826/2597052465,v1:172.21.15.46:6827/2597052465]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-06-14T22:35:21.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:21 smithi046 ceph-mon[29586]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:35:21.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:21 smithi046 ceph-mon[29586]: from='osd.3 [v2:172.21.15.46:6826/2597052465,v1:172.21.15.46:6827/2597052465]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-14T22:35:21.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:21 smithi046 ceph-mon[29586]: osdmap e24: 4 total, 3 up, 4 in 2024-06-14T22:35:21.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:21 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-14T22:35:21.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:21 smithi046 ceph-mon[29586]: from='osd.3 [v2:172.21.15.46:6826/2597052465,v1:172.21.15.46:6827/2597052465]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-06-14T22:35:21.808 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:35:21 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-3[61160]: 2024-06-14T22:35:21.505+0000 7f38561c9640 -1 osd.3 0 waiting for initial osdmap 2024-06-14T22:35:21.808 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:35:21 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-3[61160]: 2024-06-14T22:35:21.508+0000 7f38507ff640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-14T22:35:22.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:22 smithi046 ceph-mon[38286]: from='osd.3 [v2:172.21.15.46:6826/2597052465,v1:172.21.15.46:6827/2597052465]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-06-14T22:35:22.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:22 smithi046 ceph-mon[38286]: osdmap e25: 4 total, 3 up, 4 in 2024-06-14T22:35:22.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:22 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-14T22:35:22.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:22 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-14T22:35:22.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:22 smithi046 ceph-mon[29586]: from='osd.3 [v2:172.21.15.46:6826/2597052465,v1:172.21.15.46:6827/2597052465]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-06-14T22:35:22.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:22 smithi046 ceph-mon[29586]: osdmap e25: 4 total, 3 up, 4 in 2024-06-14T22:35:22.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:22 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-14T22:35:22.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:22 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-14T22:35:22.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:22 smithi178 ceph-mon[32942]: from='osd.3 [v2:172.21.15.46:6826/2597052465,v1:172.21.15.46:6827/2597052465]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-06-14T22:35:22.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:22 smithi178 ceph-mon[32942]: osdmap e25: 4 total, 3 up, 4 in 2024-06-14T22:35:22.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:22 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-14T22:35:22.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:22 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-14T22:35:22.839 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-14T22:35:22.924 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-14T22:35:22.935 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-06-14T22:35:22.935 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-06-14T22:35:22.935 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00858341 s, 1.2 GB/s 2024-06-14T22:35:22.935 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-06-14T22:35:23.538 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_4 2024-06-14T22:35:23.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:23 smithi178 ceph-mon[32942]: purged_snaps scrub starts 2024-06-14T22:35:23.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:23 smithi178 ceph-mon[32942]: purged_snaps scrub ok 2024-06-14T22:35:23.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:23 smithi178 ceph-mon[32942]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:35:23.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:23 smithi178 ceph-mon[32942]: osd.3 [v2:172.21.15.46:6826/2597052465,v1:172.21.15.46:6827/2597052465] boot 2024-06-14T22:35:23.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:23 smithi178 ceph-mon[32942]: osdmap e26: 4 total, 4 up, 4 in 2024-06-14T22:35:23.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:23 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-14T22:35:23.782 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.b/config 2024-06-14T22:35:23.848 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:23 smithi046 ceph-mon[29586]: purged_snaps scrub starts 2024-06-14T22:35:23.848 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:23 smithi046 ceph-mon[29586]: purged_snaps scrub ok 2024-06-14T22:35:23.848 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:23 smithi046 ceph-mon[29586]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:35:23.848 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:23 smithi046 ceph-mon[29586]: osd.3 [v2:172.21.15.46:6826/2597052465,v1:172.21.15.46:6827/2597052465] boot 2024-06-14T22:35:23.848 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:23 smithi046 ceph-mon[29586]: osdmap e26: 4 total, 4 up, 4 in 2024-06-14T22:35:23.848 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:23 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-14T22:35:23.849 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:23 smithi046 ceph-mon[38286]: purged_snaps scrub starts 2024-06-14T22:35:23.849 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:23 smithi046 ceph-mon[38286]: purged_snaps scrub ok 2024-06-14T22:35:23.849 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:23 smithi046 ceph-mon[38286]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-06-14T22:35:23.849 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:23 smithi046 ceph-mon[38286]: osd.3 [v2:172.21.15.46:6826/2597052465,v1:172.21.15.46:6827/2597052465] boot 2024-06-14T22:35:23.849 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:23 smithi046 ceph-mon[38286]: osdmap e26: 4 total, 4 up, 4 in 2024-06-14T22:35:23.849 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:23 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-14T22:35:24.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:24 smithi178 ceph-mon[32942]: osdmap e27: 4 total, 4 up, 4 in 2024-06-14T22:35:24.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:24.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:24.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:24.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:24.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:24.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:24.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:24.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:24.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:35:24.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:24 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:24.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[29586]: osdmap e27: 4 total, 4 up, 4 in 2024-06-14T22:35:24.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:24.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:24.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:24.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:24.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:24.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:24.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:24.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:24.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:35:24.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:24.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[38286]: osdmap e27: 4 total, 4 up, 4 in 2024-06-14T22:35:24.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:24.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:24.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:24.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:24.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:24.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:24.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:24.945 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:24.945 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:35:24.945 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:24 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:25.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:25 smithi178 ceph-mon[32942]: Detected new or changed devices on smithi046 2024-06-14T22:35:25.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:25 smithi178 ceph-mon[32942]: Adjusting osd_memory_target on smithi046 to 4017M 2024-06-14T22:35:25.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:25 smithi178 ceph-mon[32942]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:25.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:25 smithi046 ceph-mon[29586]: Detected new or changed devices on smithi046 2024-06-14T22:35:25.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:25 smithi046 ceph-mon[29586]: Adjusting osd_memory_target on smithi046 to 4017M 2024-06-14T22:35:25.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:25 smithi046 ceph-mon[29586]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:25.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:25 smithi046 ceph-mon[38286]: Detected new or changed devices on smithi046 2024-06-14T22:35:25.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:25 smithi046 ceph-mon[38286]: Adjusting osd_memory_target on smithi046 to 4017M 2024-06-14T22:35:25.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:25 smithi046 ceph-mon[38286]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:26.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:26 smithi178 ceph-mon[32942]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:35:26.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:26 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:35:26.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:26 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:35:26.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:26 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:26.942 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:26 smithi046 ceph-mon[38286]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:35:26.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:26 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:35:26.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:26 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:35:26.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:26 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:26.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:26 smithi046 ceph-mon[29586]: from='client.24208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:35:26.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:26 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:35:26.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:26 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:35:26.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:26 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:27.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:27 smithi178 ceph-mon[32942]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:27.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:27 smithi046 ceph-mon[29586]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:27.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:27 smithi046 ceph-mon[38286]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:28.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:28 smithi178 ceph-mon[32942]: from='client.? 172.21.15.178:0/3669971870' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a0b2d3d-3e61-47be-a95f-8294f79ecc7b"}]: dispatch 2024-06-14T22:35:28.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:28 smithi178 ceph-mon[32942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a0b2d3d-3e61-47be-a95f-8294f79ecc7b"}]: dispatch 2024-06-14T22:35:28.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:28 smithi178 ceph-mon[32942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a0b2d3d-3e61-47be-a95f-8294f79ecc7b"}]': finished 2024-06-14T22:35:28.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:28 smithi178 ceph-mon[32942]: osdmap e28: 5 total, 4 up, 5 in 2024-06-14T22:35:28.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:28 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:35:28.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:28 smithi178 ceph-mon[32942]: from='client.? 172.21.15.178:0/666665598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:35:28.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:28 smithi046 ceph-mon[29586]: from='client.? 172.21.15.178:0/3669971870' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a0b2d3d-3e61-47be-a95f-8294f79ecc7b"}]: dispatch 2024-06-14T22:35:28.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:28 smithi046 ceph-mon[29586]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a0b2d3d-3e61-47be-a95f-8294f79ecc7b"}]: dispatch 2024-06-14T22:35:28.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:28 smithi046 ceph-mon[29586]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a0b2d3d-3e61-47be-a95f-8294f79ecc7b"}]': finished 2024-06-14T22:35:28.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:28 smithi046 ceph-mon[29586]: osdmap e28: 5 total, 4 up, 5 in 2024-06-14T22:35:28.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:28 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:35:28.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:28 smithi046 ceph-mon[29586]: from='client.? 172.21.15.178:0/666665598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:35:28.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:28 smithi046 ceph-mon[38286]: from='client.? 172.21.15.178:0/3669971870' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a0b2d3d-3e61-47be-a95f-8294f79ecc7b"}]: dispatch 2024-06-14T22:35:28.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:28 smithi046 ceph-mon[38286]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0a0b2d3d-3e61-47be-a95f-8294f79ecc7b"}]: dispatch 2024-06-14T22:35:28.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:28 smithi046 ceph-mon[38286]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0a0b2d3d-3e61-47be-a95f-8294f79ecc7b"}]': finished 2024-06-14T22:35:28.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:28 smithi046 ceph-mon[38286]: osdmap e28: 5 total, 4 up, 5 in 2024-06-14T22:35:28.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:28 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:35:28.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:28 smithi046 ceph-mon[38286]: from='client.? 172.21.15.178:0/666665598' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:35:29.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:29 smithi178 ceph-mon[32942]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:29.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:29 smithi046 ceph-mon[29586]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:29.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:29 smithi046 ceph-mon[38286]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:31.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:31 smithi178 ceph-mon[32942]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:31.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:31 smithi046 ceph-mon[29586]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:31.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:31 smithi046 ceph-mon[38286]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:33.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:33 smithi178 ceph-mon[32942]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:33.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:33 smithi046 ceph-mon[29586]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:33.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:33 smithi046 ceph-mon[38286]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:34.867 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:34 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-14T22:35:34.867 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:34 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:34.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:34 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-14T22:35:34.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:34 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:34.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-14T22:35:34.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:35.621 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:35 smithi178 ceph-mon[32942]: Deploying daemon osd.4 on smithi178 2024-06-14T22:35:35.621 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:35 smithi178 ceph-mon[32942]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:35.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:35 smithi046 ceph-mon[29586]: Deploying daemon osd.4 on smithi178 2024-06-14T22:35:35.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:35 smithi046 ceph-mon[29586]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:35.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:35 smithi046 ceph-mon[38286]: Deploying daemon osd.4 on smithi178 2024-06-14T22:35:35.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:35 smithi046 ceph-mon[38286]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:37.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:37 smithi178 ceph-mon[32942]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:37.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:37 smithi046 ceph-mon[29586]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:37.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:37 smithi046 ceph-mon[38286]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:39.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:39 smithi178 ceph-mon[32942]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:39.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:39 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:35:39.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:39 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:39.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:39 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:39.942 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:39 smithi046 ceph-mon[38286]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:39.942 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:39 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:35:39.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:39 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:39.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:39 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:39.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:39 smithi046 ceph-mon[29586]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:39.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:39 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:35:39.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:39 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:39.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:39 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:41.720 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:41 smithi178 ceph-mon[32942]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:41.755 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 4 on host 'smithi178' 2024-06-14T22:35:41.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:41 smithi046 ceph-mon[29586]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:41.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:41 smithi046 ceph-mon[38286]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:42.456 DEBUG:teuthology.orchestra.run.smithi178:osd.4> sudo journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.4.service 2024-06-14T22:35:42.458 INFO:tasks.cephadm:Deploying osd.5 on smithi178 with /dev/vg_nvme/lv_3... 2024-06-14T22:35:42.459 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-14T22:35:42.750 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.b/config 2024-06-14T22:35:43.001 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:42 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:43.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:42 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:43.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:42 smithi178 ceph-mon[32942]: from='osd.4 [v2:172.21.15.178:6800/2179289524,v1:172.21.15.178:6801/2179289524]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-14T22:35:43.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:42 smithi178 ceph-mon[32942]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-14T22:35:43.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:42 smithi178 ceph-mon[32942]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:43.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:42 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:43.002 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:42 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:43.003 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:42 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:43.003 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:42 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:43.003 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:42 smithi046 ceph-mon[38286]: from='osd.4 [v2:172.21.15.178:6800/2179289524,v1:172.21.15.178:6801/2179289524]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-14T22:35:43.003 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:42 smithi046 ceph-mon[38286]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-14T22:35:43.003 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:42 smithi046 ceph-mon[38286]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:43.003 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:42 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:43.003 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:42 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:43.003 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:42 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:43.003 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:42 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:43.004 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:42 smithi046 ceph-mon[29586]: from='osd.4 [v2:172.21.15.178:6800/2179289524,v1:172.21.15.178:6801/2179289524]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-14T22:35:43.004 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:42 smithi046 ceph-mon[29586]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-14T22:35:43.004 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:42 smithi046 ceph-mon[29586]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:43.004 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:42 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:43.004 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:42 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:44.024 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:43 smithi178 ceph-mon[32942]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-14T22:35:44.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:43 smithi178 ceph-mon[32942]: osdmap e29: 5 total, 4 up, 5 in 2024-06-14T22:35:44.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:43 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:35:44.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:43 smithi178 ceph-mon[32942]: from='osd.4 [v2:172.21.15.178:6800/2179289524,v1:172.21.15.178:6801/2179289524]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:35:44.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:43 smithi178 ceph-mon[32942]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:35:44.025 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:35:43 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4[38354]: 2024-06-14T22:35:43.769+0000 7efefbc4b640 -1 osd.4 0 waiting for initial osdmap 2024-06-14T22:35:44.025 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:35:43 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4[38354]: 2024-06-14T22:35:43.773+0000 7efef6a60640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-14T22:35:44.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:43 smithi046 ceph-mon[38286]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-14T22:35:44.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:43 smithi046 ceph-mon[38286]: osdmap e29: 5 total, 4 up, 5 in 2024-06-14T22:35:44.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:43 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:35:44.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:43 smithi046 ceph-mon[38286]: from='osd.4 [v2:172.21.15.178:6800/2179289524,v1:172.21.15.178:6801/2179289524]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:35:44.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:43 smithi046 ceph-mon[38286]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:35:44.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:43 smithi046 ceph-mon[29586]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-14T22:35:44.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:43 smithi046 ceph-mon[29586]: osdmap e29: 5 total, 4 up, 5 in 2024-06-14T22:35:44.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:43 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:35:44.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:43 smithi046 ceph-mon[29586]: from='osd.4 [v2:172.21.15.178:6800/2179289524,v1:172.21.15.178:6801/2179289524]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:35:44.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:43 smithi046 ceph-mon[29586]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:35:45.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:44 smithi178 ceph-mon[32942]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-06-14T22:35:45.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:44 smithi178 ceph-mon[32942]: osdmap e30: 5 total, 4 up, 5 in 2024-06-14T22:35:45.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:44 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:35:45.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:44 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:35:45.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:44 smithi178 ceph-mon[32942]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:45.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:44 smithi046 ceph-mon[29586]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-06-14T22:35:45.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:44 smithi046 ceph-mon[29586]: osdmap e30: 5 total, 4 up, 5 in 2024-06-14T22:35:45.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:44 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:35:45.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:44 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:35:45.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:44 smithi046 ceph-mon[29586]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:45.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:44 smithi046 ceph-mon[38286]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-06-14T22:35:45.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:44 smithi046 ceph-mon[38286]: osdmap e30: 5 total, 4 up, 5 in 2024-06-14T22:35:45.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:44 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:35:45.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:44 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:35:45.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:44 smithi046 ceph-mon[38286]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-06-14T22:35:45.952 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:45 smithi178 ceph-mon[32942]: purged_snaps scrub starts 2024-06-14T22:35:45.953 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:45 smithi178 ceph-mon[32942]: purged_snaps scrub ok 2024-06-14T22:35:45.953 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:45 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:35:45.953 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:45 smithi178 ceph-mon[32942]: osd.4 [v2:172.21.15.178:6800/2179289524,v1:172.21.15.178:6801/2179289524] boot 2024-06-14T22:35:45.953 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:45 smithi178 ceph-mon[32942]: osdmap e31: 5 total, 5 up, 5 in 2024-06-14T22:35:45.953 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:45 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:35:45.961 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-14T22:35:46.050 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-14T22:35:46.062 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-06-14T22:35:46.062 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-06-14T22:35:46.062 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0096457 s, 1.1 GB/s 2024-06-14T22:35:46.062 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-06-14T22:35:46.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:45 smithi046 ceph-mon[29586]: purged_snaps scrub starts 2024-06-14T22:35:46.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:45 smithi046 ceph-mon[29586]: purged_snaps scrub ok 2024-06-14T22:35:46.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:45 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:35:46.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:45 smithi046 ceph-mon[29586]: osd.4 [v2:172.21.15.178:6800/2179289524,v1:172.21.15.178:6801/2179289524] boot 2024-06-14T22:35:46.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:45 smithi046 ceph-mon[29586]: osdmap e31: 5 total, 5 up, 5 in 2024-06-14T22:35:46.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:45 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:35:46.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:45 smithi046 ceph-mon[38286]: purged_snaps scrub starts 2024-06-14T22:35:46.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:45 smithi046 ceph-mon[38286]: purged_snaps scrub ok 2024-06-14T22:35:46.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:45 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:35:46.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:45 smithi046 ceph-mon[38286]: osd.4 [v2:172.21.15.178:6800/2179289524,v1:172.21.15.178:6801/2179289524] boot 2024-06-14T22:35:46.194 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:45 smithi046 ceph-mon[38286]: osdmap e31: 5 total, 5 up, 5 in 2024-06-14T22:35:46.194 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:45 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:35:46.705 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_3 2024-06-14T22:35:47.009 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:46 smithi178 ceph-mon[32942]: osdmap e32: 5 total, 5 up, 5 in 2024-06-14T22:35:47.009 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:46 smithi178 ceph-mon[32942]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:35:47.009 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:46 smithi178 ceph-mon[32942]: osdmap e33: 5 total, 5 up, 5 in 2024-06-14T22:35:47.192 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:46 smithi046 ceph-mon[29586]: osdmap e32: 5 total, 5 up, 5 in 2024-06-14T22:35:47.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:46 smithi046 ceph-mon[29586]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:35:47.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:46 smithi046 ceph-mon[29586]: osdmap e33: 5 total, 5 up, 5 in 2024-06-14T22:35:47.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:46 smithi046 ceph-mon[38286]: osdmap e32: 5 total, 5 up, 5 in 2024-06-14T22:35:47.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:46 smithi046 ceph-mon[38286]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:35:47.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:46 smithi046 ceph-mon[38286]: osdmap e33: 5 total, 5 up, 5 in 2024-06-14T22:35:47.297 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.b/config 2024-06-14T22:35:48.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:48 smithi178 ceph-mon[32942]: Detected new or changed devices on smithi178 2024-06-14T22:35:48.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:48 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:48.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:48 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:48.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:48 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:48.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:48 smithi178 ceph-mon[32942]: Adjusting osd_memory_target on smithi178 to 17088M 2024-06-14T22:35:48.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:48 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:48.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:48 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:48.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:48 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:35:48.526 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:48 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:48.526 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:48 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:35:48.526 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:48 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:48.526 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:48 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:35:48.526 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:48 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:48.526 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:48 smithi178 ceph-mon[32942]: osdmap e34: 5 total, 5 up, 5 in 2024-06-14T22:35:48.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[38286]: Detected new or changed devices on smithi178 2024-06-14T22:35:48.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:48.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:48.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:48.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[38286]: Adjusting osd_memory_target on smithi178 to 17088M 2024-06-14T22:35:48.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:48.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:48.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:35:48.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:48.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:35:48.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:48.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:35:48.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:48.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[38286]: osdmap e34: 5 total, 5 up, 5 in 2024-06-14T22:35:48.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[29586]: Detected new or changed devices on smithi178 2024-06-14T22:35:48.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:48.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:48.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:35:48.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[29586]: Adjusting osd_memory_target on smithi178 to 17088M 2024-06-14T22:35:48.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:48.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:48.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:35:48.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:48.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:35:48.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:48.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:35:48.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:35:48.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:48 smithi046 ceph-mon[29586]: osdmap e34: 5 total, 5 up, 5 in 2024-06-14T22:35:49.617 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:49 smithi178 ceph-mon[32942]: pgmap v116: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-14T22:35:49.617 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:49 smithi178 ceph-mon[32942]: osdmap e35: 5 total, 5 up, 5 in 2024-06-14T22:35:49.617 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:49 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:35:49.618 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:49 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:35:49.618 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:49 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:49.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:49 smithi046 ceph-mon[29586]: pgmap v116: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-14T22:35:49.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:49 smithi046 ceph-mon[29586]: osdmap e35: 5 total, 5 up, 5 in 2024-06-14T22:35:49.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:49 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:35:49.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:49 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:35:49.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:49 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:49.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:49 smithi046 ceph-mon[38286]: pgmap v116: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-14T22:35:49.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:49 smithi046 ceph-mon[38286]: osdmap e35: 5 total, 5 up, 5 in 2024-06-14T22:35:49.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:49 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:35:49.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:49 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:35:49.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:49 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:50.513 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:50 smithi178 ceph-mon[32942]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:35:50.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:50 smithi046 ceph-mon[29586]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:35:50.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:50 smithi046 ceph-mon[38286]: from='client.24235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:35:51.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:51 smithi178 ceph-mon[32942]: pgmap v118: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-06-14T22:35:51.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:51 smithi046 ceph-mon[29586]: pgmap v118: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-06-14T22:35:51.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:51 smithi046 ceph-mon[38286]: pgmap v118: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-06-14T22:35:52.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:52 smithi046 ceph-mon[29586]: from='client.? 172.21.15.178:0/478960308' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29f79df4-039f-44a3-8570-7252eca3abe8"}]: dispatch 2024-06-14T22:35:52.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:52 smithi046 ceph-mon[29586]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29f79df4-039f-44a3-8570-7252eca3abe8"}]: dispatch 2024-06-14T22:35:52.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:52 smithi046 ceph-mon[29586]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29f79df4-039f-44a3-8570-7252eca3abe8"}]': finished 2024-06-14T22:35:52.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:52 smithi046 ceph-mon[29586]: osdmap e36: 6 total, 5 up, 6 in 2024-06-14T22:35:52.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:52 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-14T22:35:52.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:52 smithi046 ceph-mon[29586]: from='client.? 172.21.15.178:0/2789630024' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:35:52.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:52 smithi046 ceph-mon[38286]: from='client.? 172.21.15.178:0/478960308' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29f79df4-039f-44a3-8570-7252eca3abe8"}]: dispatch 2024-06-14T22:35:52.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:52 smithi046 ceph-mon[38286]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29f79df4-039f-44a3-8570-7252eca3abe8"}]: dispatch 2024-06-14T22:35:52.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:52 smithi046 ceph-mon[38286]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29f79df4-039f-44a3-8570-7252eca3abe8"}]': finished 2024-06-14T22:35:52.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:52 smithi046 ceph-mon[38286]: osdmap e36: 6 total, 5 up, 6 in 2024-06-14T22:35:52.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:52 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-14T22:35:52.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:52 smithi046 ceph-mon[38286]: from='client.? 172.21.15.178:0/2789630024' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:35:52.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:52 smithi178 ceph-mon[32942]: from='client.? 172.21.15.178:0/478960308' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29f79df4-039f-44a3-8570-7252eca3abe8"}]: dispatch 2024-06-14T22:35:52.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:52 smithi178 ceph-mon[32942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "29f79df4-039f-44a3-8570-7252eca3abe8"}]: dispatch 2024-06-14T22:35:52.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:52 smithi178 ceph-mon[32942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "29f79df4-039f-44a3-8570-7252eca3abe8"}]': finished 2024-06-14T22:35:52.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:52 smithi178 ceph-mon[32942]: osdmap e36: 6 total, 5 up, 6 in 2024-06-14T22:35:52.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:52 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-14T22:35:52.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:52 smithi178 ceph-mon[32942]: from='client.? 172.21.15.178:0/2789630024' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:35:53.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:53 smithi046 ceph-mon[29586]: pgmap v120: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-06-14T22:35:53.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:53 smithi046 ceph-mon[38286]: pgmap v120: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-06-14T22:35:53.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:53 smithi178 ceph-mon[32942]: pgmap v120: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-06-14T22:35:55.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:55 smithi178 ceph-mon[32942]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:35:55.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:55 smithi046 ceph-mon[29586]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:35:55.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:55 smithi046 ceph-mon[38286]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:35:57.599 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:57 smithi178 ceph-mon[32942]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:35:57.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:57 smithi046 ceph-mon[29586]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:35:57.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:57 smithi046 ceph-mon[38286]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:35:58.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:58 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-14T22:35:58.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:58 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:58.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:58 smithi046 ceph-mon[29586]: Deploying daemon osd.5 on smithi178 2024-06-14T22:35:58.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:58 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-14T22:35:58.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:58 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:58.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:58 smithi046 ceph-mon[38286]: Deploying daemon osd.5 on smithi178 2024-06-14T22:35:58.747 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:58 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-14T22:35:58.747 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:58 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:35:58.747 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:58 smithi178 ceph-mon[32942]: Deploying daemon osd.5 on smithi178 2024-06-14T22:35:59.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:35:59 smithi046 ceph-mon[29586]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:35:59.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:35:59 smithi046 ceph-mon[38286]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:35:59.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:35:59 smithi178 ceph-mon[32942]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:36:01.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:01 smithi046 ceph-mon[29586]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:36:01.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:01 smithi046 ceph-mon[38286]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:36:01.745 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:01 smithi178 ceph-mon[32942]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:36:03.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:03 smithi046 ceph-mon[29586]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:36:03.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:03 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:36:03.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:03 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:03.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:03 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:03.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:03 smithi046 ceph-mon[38286]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:36:03.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:03 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:36:03.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:03 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:03.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:03 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:03.754 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:03 smithi178 ceph-mon[32942]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:36:03.755 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:03 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:36:03.755 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:03 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:03.755 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:03 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:05.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:05 smithi046 ceph-mon[29586]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:36:05.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:05 smithi046 ceph-mon[38286]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:36:05.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:05 smithi178 ceph-mon[32942]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:36:05.832 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 5 on host 'smithi178' 2024-06-14T22:36:06.469 DEBUG:teuthology.orchestra.run.smithi178:osd.5> sudo journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.5.service 2024-06-14T22:36:06.472 INFO:tasks.cephadm:Deploying osd.6 on smithi178 with /dev/vg_nvme/lv_2... 2024-06-14T22:36:06.472 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-14T22:36:06.714 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:06 smithi178 ceph-mon[32942]: from='osd.5 [v2:172.21.15.178:6808/2813027226,v1:172.21.15.178:6809/2813027226]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-14T22:36:06.714 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:06 smithi178 ceph-mon[32942]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-14T22:36:06.715 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:06 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:06.715 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:06 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:06.715 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:06 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:06.715 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:06 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:06.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:06 smithi046 ceph-mon[29586]: from='osd.5 [v2:172.21.15.178:6808/2813027226,v1:172.21.15.178:6809/2813027226]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-14T22:36:06.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:06 smithi046 ceph-mon[29586]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-14T22:36:06.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:06 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:06.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:06 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:06.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:06 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:06.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:06 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:06.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:06 smithi046 ceph-mon[38286]: from='osd.5 [v2:172.21.15.178:6808/2813027226,v1:172.21.15.178:6809/2813027226]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-14T22:36:06.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:06 smithi046 ceph-mon[38286]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-14T22:36:06.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:06 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:06.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:06 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:06.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:06 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:06.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:06 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:07.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:07 smithi178 ceph-mon[32942]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:36:07.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:07 smithi178 ceph-mon[32942]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-14T22:36:07.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:07 smithi178 ceph-mon[32942]: osdmap e37: 6 total, 5 up, 6 in 2024-06-14T22:36:07.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:07 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-14T22:36:07.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:07 smithi178 ceph-mon[32942]: from='osd.5 [v2:172.21.15.178:6808/2813027226,v1:172.21.15.178:6809/2813027226]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:36:07.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:07 smithi178 ceph-mon[32942]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:36:07.525 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:36:07 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:36:07.456+0000 7fdf82a8e640 -1 osd.5 0 waiting for initial osdmap 2024-06-14T22:36:07.525 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:36:07 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:36:07.461+0000 7fdf7e8c7640 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-14T22:36:07.796 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.b/config 2024-06-14T22:36:07.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:07 smithi046 ceph-mon[29586]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:36:07.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:07 smithi046 ceph-mon[29586]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-14T22:36:07.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:07 smithi046 ceph-mon[29586]: osdmap e37: 6 total, 5 up, 6 in 2024-06-14T22:36:07.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:07 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-14T22:36:07.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:07 smithi046 ceph-mon[29586]: from='osd.5 [v2:172.21.15.178:6808/2813027226,v1:172.21.15.178:6809/2813027226]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:36:07.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:07 smithi046 ceph-mon[29586]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:36:07.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:07 smithi046 ceph-mon[38286]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:36:07.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:07 smithi046 ceph-mon[38286]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-14T22:36:07.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:07 smithi046 ceph-mon[38286]: osdmap e37: 6 total, 5 up, 6 in 2024-06-14T22:36:07.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:07 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-14T22:36:07.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:07 smithi046 ceph-mon[38286]: from='osd.5 [v2:172.21.15.178:6808/2813027226,v1:172.21.15.178:6809/2813027226]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:36:07.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:07 smithi046 ceph-mon[38286]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:36:08.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:08 smithi178 ceph-mon[32942]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-06-14T22:36:08.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:08 smithi178 ceph-mon[32942]: osdmap e38: 6 total, 5 up, 6 in 2024-06-14T22:36:08.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:08 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-14T22:36:08.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:08 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-14T22:36:08.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:08 smithi046 ceph-mon[29586]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-06-14T22:36:08.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:08 smithi046 ceph-mon[29586]: osdmap e38: 6 total, 5 up, 6 in 2024-06-14T22:36:08.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:08 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-14T22:36:08.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:08 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-14T22:36:08.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:08 smithi046 ceph-mon[38286]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-06-14T22:36:08.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:08 smithi046 ceph-mon[38286]: osdmap e38: 6 total, 5 up, 6 in 2024-06-14T22:36:08.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:08 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-14T22:36:08.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:08 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-14T22:36:09.627 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:09 smithi178 ceph-mon[32942]: purged_snaps scrub starts 2024-06-14T22:36:09.628 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:09 smithi178 ceph-mon[32942]: purged_snaps scrub ok 2024-06-14T22:36:09.628 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:09 smithi178 ceph-mon[32942]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:36:09.628 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:09 smithi178 ceph-mon[32942]: osd.5 [v2:172.21.15.178:6808/2813027226,v1:172.21.15.178:6809/2813027226] boot 2024-06-14T22:36:09.628 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:09 smithi178 ceph-mon[32942]: osdmap e39: 6 total, 6 up, 6 in 2024-06-14T22:36:09.628 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:09 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-14T22:36:09.628 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:09 smithi178 ceph-mon[32942]: osdmap e40: 6 total, 6 up, 6 in 2024-06-14T22:36:09.942 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:09 smithi046 ceph-mon[38286]: purged_snaps scrub starts 2024-06-14T22:36:09.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:09 smithi046 ceph-mon[38286]: purged_snaps scrub ok 2024-06-14T22:36:09.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:09 smithi046 ceph-mon[38286]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:36:09.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:09 smithi046 ceph-mon[38286]: osd.5 [v2:172.21.15.178:6808/2813027226,v1:172.21.15.178:6809/2813027226] boot 2024-06-14T22:36:09.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:09 smithi046 ceph-mon[38286]: osdmap e39: 6 total, 6 up, 6 in 2024-06-14T22:36:09.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:09 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-14T22:36:09.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:09 smithi046 ceph-mon[38286]: osdmap e40: 6 total, 6 up, 6 in 2024-06-14T22:36:09.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:09 smithi046 ceph-mon[29586]: purged_snaps scrub starts 2024-06-14T22:36:09.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:09 smithi046 ceph-mon[29586]: purged_snaps scrub ok 2024-06-14T22:36:09.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:09 smithi046 ceph-mon[29586]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-06-14T22:36:09.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:09 smithi046 ceph-mon[29586]: osd.5 [v2:172.21.15.178:6808/2813027226,v1:172.21.15.178:6809/2813027226] boot 2024-06-14T22:36:09.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:09 smithi046 ceph-mon[29586]: osdmap e39: 6 total, 6 up, 6 in 2024-06-14T22:36:09.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:09 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-14T22:36:09.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:09 smithi046 ceph-mon[29586]: osdmap e40: 6 total, 6 up, 6 in 2024-06-14T22:36:10.700 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-14T22:36:10.785 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-14T22:36:10.797 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-06-14T22:36:10.797 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-06-14T22:36:10.797 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00950777 s, 1.1 GB/s 2024-06-14T22:36:10.797 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-06-14T22:36:11.466 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:11 smithi178 ceph-mon[32942]: Detected new or changed devices on smithi178 2024-06-14T22:36:11.466 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:11 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:11.466 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:11 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:11.467 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:11 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:36:11.467 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:11 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:36:11.467 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:11 smithi178 ceph-mon[32942]: Adjusting osd_memory_target on smithi178 to 8544M 2024-06-14T22:36:11.467 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:11 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:11.467 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:11 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:11.467 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:11 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:36:11.467 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:11 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:11.467 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:11 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:36:11.467 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:11 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:11.467 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:11 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:36:11.467 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:11 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:11.467 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:11 smithi178 ceph-mon[32942]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:11.468 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:11 smithi178 ceph-mon[32942]: osdmap e41: 6 total, 6 up, 6 in 2024-06-14T22:36:11.468 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:11 smithi178 ceph-mon[32942]: osdmap e42: 6 total, 6 up, 6 in 2024-06-14T22:36:11.492 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_2 2024-06-14T22:36:11.629 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[38286]: Detected new or changed devices on smithi178 2024-06-14T22:36:11.630 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:11.630 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:11.630 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:36:11.630 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:36:11.630 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[38286]: Adjusting osd_memory_target on smithi178 to 8544M 2024-06-14T22:36:11.630 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:11.630 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:11.630 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:36:11.630 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:11.630 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:36:11.630 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:11.631 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:36:11.631 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:11.631 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[38286]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:11.631 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[38286]: osdmap e41: 6 total, 6 up, 6 in 2024-06-14T22:36:11.631 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[38286]: osdmap e42: 6 total, 6 up, 6 in 2024-06-14T22:36:11.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[29586]: Detected new or changed devices on smithi178 2024-06-14T22:36:11.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:11.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:11.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:36:11.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:36:11.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[29586]: Adjusting osd_memory_target on smithi178 to 8544M 2024-06-14T22:36:11.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:11.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:11.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:36:11.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:11.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:36:11.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:11.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:36:11.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:11.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[29586]: pgmap v133: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:11.633 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[29586]: osdmap e41: 6 total, 6 up, 6 in 2024-06-14T22:36:11.633 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:11 smithi046 ceph-mon[29586]: osdmap e42: 6 total, 6 up, 6 in 2024-06-14T22:36:11.741 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.b/config 2024-06-14T22:36:12.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:12 smithi178 ceph-mon[32942]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-14T22:36:12.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:12 smithi178 ceph-mon[32942]: osdmap e43: 6 total, 6 up, 6 in 2024-06-14T22:36:12.597 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:12 smithi046 ceph-mon[29586]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-14T22:36:12.597 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:12 smithi046 ceph-mon[29586]: osdmap e43: 6 total, 6 up, 6 in 2024-06-14T22:36:12.597 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:12 smithi046 ceph-mon[38286]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-06-14T22:36:12.597 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:12 smithi046 ceph-mon[38286]: osdmap e43: 6 total, 6 up, 6 in 2024-06-14T22:36:13.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:13 smithi178 ceph-mon[32942]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:13.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:13 smithi046 ceph-mon[29586]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:13.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:13 smithi046 ceph-mon[38286]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:14.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:14 smithi178 ceph-mon[32942]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:36:14.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:14 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:36:14.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:14 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:36:14.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:14 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:14.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:14 smithi046 ceph-mon[29586]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:36:14.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:14 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:36:14.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:14 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:36:14.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:14 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:14.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:14 smithi046 ceph-mon[38286]: from='client.24262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:36:14.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:14 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:36:14.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:14 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:36:14.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:14 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:15.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:15 smithi178 ceph-mon[32942]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-06-14T22:36:15.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:15 smithi046 ceph-mon[29586]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-06-14T22:36:15.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:15 smithi046 ceph-mon[38286]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-06-14T22:36:16.513 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:16 smithi178 ceph-mon[32942]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-14T22:36:16.513 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:16 smithi178 ceph-mon[32942]: Cluster is now healthy 2024-06-14T22:36:16.513 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:16 smithi178 ceph-mon[32942]: from='client.? 172.21.15.178:0/706333520' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56661a2b-4e13-47e7-972a-bb1861d5e33d"}]: dispatch 2024-06-14T22:36:16.514 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:16 smithi178 ceph-mon[32942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56661a2b-4e13-47e7-972a-bb1861d5e33d"}]: dispatch 2024-06-14T22:36:16.514 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:16 smithi178 ceph-mon[32942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56661a2b-4e13-47e7-972a-bb1861d5e33d"}]': finished 2024-06-14T22:36:16.514 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:16 smithi178 ceph-mon[32942]: osdmap e44: 7 total, 6 up, 7 in 2024-06-14T22:36:16.514 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:16 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:36:16.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:16 smithi046 ceph-mon[29586]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-14T22:36:16.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:16 smithi046 ceph-mon[29586]: Cluster is now healthy 2024-06-14T22:36:16.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:16 smithi046 ceph-mon[29586]: from='client.? 172.21.15.178:0/706333520' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56661a2b-4e13-47e7-972a-bb1861d5e33d"}]: dispatch 2024-06-14T22:36:16.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:16 smithi046 ceph-mon[29586]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56661a2b-4e13-47e7-972a-bb1861d5e33d"}]: dispatch 2024-06-14T22:36:16.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:16 smithi046 ceph-mon[29586]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56661a2b-4e13-47e7-972a-bb1861d5e33d"}]': finished 2024-06-14T22:36:16.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:16 smithi046 ceph-mon[29586]: osdmap e44: 7 total, 6 up, 7 in 2024-06-14T22:36:16.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:16 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:36:16.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:16 smithi046 ceph-mon[38286]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-06-14T22:36:16.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:16 smithi046 ceph-mon[38286]: Cluster is now healthy 2024-06-14T22:36:16.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:16 smithi046 ceph-mon[38286]: from='client.? 172.21.15.178:0/706333520' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56661a2b-4e13-47e7-972a-bb1861d5e33d"}]: dispatch 2024-06-14T22:36:16.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:16 smithi046 ceph-mon[38286]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "56661a2b-4e13-47e7-972a-bb1861d5e33d"}]: dispatch 2024-06-14T22:36:16.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:16 smithi046 ceph-mon[38286]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "56661a2b-4e13-47e7-972a-bb1861d5e33d"}]': finished 2024-06-14T22:36:16.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:16 smithi046 ceph-mon[38286]: osdmap e44: 7 total, 6 up, 7 in 2024-06-14T22:36:16.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:16 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:36:17.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:17 smithi178 ceph-mon[32942]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-06-14T22:36:17.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:17 smithi178 ceph-mon[32942]: from='client.? 172.21.15.178:0/2742752117' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:36:17.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:17 smithi046 ceph-mon[29586]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-06-14T22:36:17.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:17 smithi046 ceph-mon[29586]: from='client.? 172.21.15.178:0/2742752117' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:36:17.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:17 smithi046 ceph-mon[38286]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-06-14T22:36:17.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:17 smithi046 ceph-mon[38286]: from='client.? 172.21.15.178:0/2742752117' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:36:19.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:19 smithi046 ceph-mon[29586]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-06-14T22:36:19.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:19 smithi046 ceph-mon[38286]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-06-14T22:36:19.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:19 smithi178 ceph-mon[32942]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-06-14T22:36:21.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:21 smithi046 ceph-mon[29586]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-06-14T22:36:21.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:21 smithi046 ceph-mon[38286]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-06-14T22:36:21.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:21 smithi178 ceph-mon[32942]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-06-14T22:36:22.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:22 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-14T22:36:22.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:22 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:22.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:22 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-14T22:36:22.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:22 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:22.704 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:22 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-14T22:36:22.704 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:22 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:23.561 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:23 smithi178 ceph-mon[32942]: Deploying daemon osd.6 on smithi178 2024-06-14T22:36:23.561 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:23 smithi178 ceph-mon[32942]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-14T22:36:23.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:23 smithi046 ceph-mon[29586]: Deploying daemon osd.6 on smithi178 2024-06-14T22:36:23.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:23 smithi046 ceph-mon[29586]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-14T22:36:23.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:23 smithi046 ceph-mon[38286]: Deploying daemon osd.6 on smithi178 2024-06-14T22:36:23.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:23 smithi046 ceph-mon[38286]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-14T22:36:25.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:25 smithi046 ceph-mon[29586]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:25.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:25 smithi046 ceph-mon[38286]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:25.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:25 smithi178 ceph-mon[32942]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:27.629 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:27 smithi178 ceph-mon[32942]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:27.630 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:27 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:36:27.630 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:27 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:27.630 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:27 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:27.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:27 smithi046 ceph-mon[29586]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:27.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:27 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:36:27.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:27 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:27.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:27 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:27.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:27 smithi046 ceph-mon[38286]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:27.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:36:27.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:27.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:27 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:29.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:29 smithi178 ceph-mon[32942]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:29.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:29 smithi046 ceph-mon[29586]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:29.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:29 smithi046 ceph-mon[38286]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:30.826 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 6 on host 'smithi178' 2024-06-14T22:36:31.455 DEBUG:teuthology.orchestra.run.smithi178:osd.6> sudo journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.6.service 2024-06-14T22:36:31.458 INFO:tasks.cephadm:Deploying osd.7 on smithi178 with /dev/vg_nvme/lv_1... 2024-06-14T22:36:31.458 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-14T22:36:31.756 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.b/config 2024-06-14T22:36:31.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:31 smithi178 ceph-mon[32942]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:31.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:31 smithi178 ceph-mon[32942]: from='osd.6 [v2:172.21.15.178:6816/3396977870,v1:172.21.15.178:6817/3396977870]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-14T22:36:31.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:31 smithi178 ceph-mon[32942]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-14T22:36:31.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:31 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:31.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:31 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:31.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:31 smithi046 ceph-mon[29586]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:31.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:31 smithi046 ceph-mon[29586]: from='osd.6 [v2:172.21.15.178:6816/3396977870,v1:172.21.15.178:6817/3396977870]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-14T22:36:31.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:31 smithi046 ceph-mon[29586]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-14T22:36:31.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:31 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:31.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:31 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:31.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:31 smithi046 ceph-mon[38286]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:31.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:31 smithi046 ceph-mon[38286]: from='osd.6 [v2:172.21.15.178:6816/3396977870,v1:172.21.15.178:6817/3396977870]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-14T22:36:31.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:31 smithi046 ceph-mon[38286]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-14T22:36:31.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:31 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:31.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:31 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:32.456 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:32 smithi178 ceph-mon[32942]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-14T22:36:32.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:32 smithi178 ceph-mon[32942]: osdmap e45: 7 total, 6 up, 7 in 2024-06-14T22:36:32.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:32 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:36:32.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:32 smithi178 ceph-mon[32942]: from='osd.6 [v2:172.21.15.178:6816/3396977870,v1:172.21.15.178:6817/3396977870]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:36:32.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:32 smithi178 ceph-mon[32942]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:36:32.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:32 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:32.457 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:32 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:32.775 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:36:32 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:36:32.463+0000 7f130aa41640 -1 osd.6 0 waiting for initial osdmap 2024-06-14T22:36:32.775 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:36:32 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:36:32.470+0000 7f1305077640 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-14T22:36:32.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:32 smithi046 ceph-mon[29586]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-14T22:36:32.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:32 smithi046 ceph-mon[29586]: osdmap e45: 7 total, 6 up, 7 in 2024-06-14T22:36:32.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:32 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:36:32.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:32 smithi046 ceph-mon[29586]: from='osd.6 [v2:172.21.15.178:6816/3396977870,v1:172.21.15.178:6817/3396977870]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:36:32.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:32 smithi046 ceph-mon[29586]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:36:32.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:32 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:32.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:32 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:32.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:32 smithi046 ceph-mon[38286]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-14T22:36:32.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:32 smithi046 ceph-mon[38286]: osdmap e45: 7 total, 6 up, 7 in 2024-06-14T22:36:32.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:32 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:36:32.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:32 smithi046 ceph-mon[38286]: from='osd.6 [v2:172.21.15.178:6816/3396977870,v1:172.21.15.178:6817/3396977870]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:36:32.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:32 smithi046 ceph-mon[38286]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:36:32.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:32 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:32.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:32 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:33.765 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-14T22:36:33.765 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:33 smithi178 ceph-mon[32942]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:33.765 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:33 smithi178 ceph-mon[32942]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-06-14T22:36:33.765 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:33 smithi178 ceph-mon[32942]: osdmap e46: 7 total, 6 up, 7 in 2024-06-14T22:36:33.765 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:33 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:36:33.765 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:33 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:36:33.844 INFO:teuthology.orchestra.run.smithi178.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-14T22:36:33.856 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10+0 records in 2024-06-14T22:36:33.856 INFO:teuthology.orchestra.run.smithi178.stderr:10+0 records out 2024-06-14T22:36:33.856 INFO:teuthology.orchestra.run.smithi178.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00980938 s, 1.1 GB/s 2024-06-14T22:36:33.856 INFO:teuthology.orchestra.run.smithi178.stderr:--> Zapping successful for: 2024-06-14T22:36:33.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:33 smithi046 ceph-mon[29586]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:33.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:33 smithi046 ceph-mon[29586]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-06-14T22:36:33.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:33 smithi046 ceph-mon[29586]: osdmap e46: 7 total, 6 up, 7 in 2024-06-14T22:36:33.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:33 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:36:33.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:33 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:36:33.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:33 smithi046 ceph-mon[38286]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:33.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:33 smithi046 ceph-mon[38286]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-06-14T22:36:33.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:33 smithi046 ceph-mon[38286]: osdmap e46: 7 total, 6 up, 7 in 2024-06-14T22:36:33.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:33 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:36:33.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:33 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:36:34.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:34 smithi178 ceph-mon[32942]: purged_snaps scrub starts 2024-06-14T22:36:34.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:34 smithi178 ceph-mon[32942]: purged_snaps scrub ok 2024-06-14T22:36:34.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:34 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:36:34.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:34 smithi178 ceph-mon[32942]: osd.6 [v2:172.21.15.178:6816/3396977870,v1:172.21.15.178:6817/3396977870] boot 2024-06-14T22:36:34.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:34 smithi178 ceph-mon[32942]: osdmap e47: 7 total, 7 up, 7 in 2024-06-14T22:36:34.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:34 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:36:34.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:34 smithi046 ceph-mon[29586]: purged_snaps scrub starts 2024-06-14T22:36:34.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:34 smithi046 ceph-mon[29586]: purged_snaps scrub ok 2024-06-14T22:36:34.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:34 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:36:34.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:34 smithi046 ceph-mon[29586]: osd.6 [v2:172.21.15.178:6816/3396977870,v1:172.21.15.178:6817/3396977870] boot 2024-06-14T22:36:34.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:34 smithi046 ceph-mon[29586]: osdmap e47: 7 total, 7 up, 7 in 2024-06-14T22:36:34.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:34 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:36:34.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:34 smithi046 ceph-mon[38286]: purged_snaps scrub starts 2024-06-14T22:36:34.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:34 smithi046 ceph-mon[38286]: purged_snaps scrub ok 2024-06-14T22:36:34.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:36:34.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:34 smithi046 ceph-mon[38286]: osd.6 [v2:172.21.15.178:6816/3396977870,v1:172.21.15.178:6817/3396977870] boot 2024-06-14T22:36:34.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:34 smithi046 ceph-mon[38286]: osdmap e47: 7 total, 7 up, 7 in 2024-06-14T22:36:34.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:34 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:36:35.058 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_1 2024-06-14T22:36:35.574 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.b/config 2024-06-14T22:36:35.698 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:35 smithi178 ceph-mon[32942]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:35.698 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:35 smithi178 ceph-mon[32942]: osdmap e48: 7 total, 7 up, 7 in 2024-06-14T22:36:35.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:35 smithi046 ceph-mon[29586]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:35.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:35 smithi046 ceph-mon[29586]: osdmap e48: 7 total, 7 up, 7 in 2024-06-14T22:36:35.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:35 smithi046 ceph-mon[38286]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-06-14T22:36:35.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:35 smithi046 ceph-mon[38286]: osdmap e48: 7 total, 7 up, 7 in 2024-06-14T22:36:36.742 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:36 smithi178 ceph-mon[32942]: osdmap e49: 7 total, 7 up, 7 in 2024-06-14T22:36:36.742 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:36 smithi178 ceph-mon[32942]: osdmap e50: 7 total, 7 up, 7 in 2024-06-14T22:36:36.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:36 smithi046 ceph-mon[29586]: osdmap e49: 7 total, 7 up, 7 in 2024-06-14T22:36:36.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:36 smithi046 ceph-mon[29586]: osdmap e50: 7 total, 7 up, 7 in 2024-06-14T22:36:36.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:36 smithi046 ceph-mon[38286]: osdmap e49: 7 total, 7 up, 7 in 2024-06-14T22:36:36.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:36 smithi046 ceph-mon[38286]: osdmap e50: 7 total, 7 up, 7 in 2024-06-14T22:36:37.747 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:37 smithi178 ceph-mon[32942]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-06-14T22:36:37.747 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:37 smithi178 ceph-mon[32942]: osdmap e51: 7 total, 7 up, 7 in 2024-06-14T22:36:37.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:37 smithi046 ceph-mon[29586]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-06-14T22:36:37.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:37 smithi046 ceph-mon[29586]: osdmap e51: 7 total, 7 up, 7 in 2024-06-14T22:36:37.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:37 smithi046 ceph-mon[38286]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-06-14T22:36:37.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:37 smithi046 ceph-mon[38286]: osdmap e51: 7 total, 7 up, 7 in 2024-06-14T22:36:39.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: Detected new or changed devices on smithi178 2024-06-14T22:36:39.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:39.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:39.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:36:39.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:36:39.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:36:39.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: Adjusting osd_memory_target on smithi178 to 5696M 2024-06-14T22:36:39.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:39.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:39.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:36:39.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:39.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:36:39.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:39.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:36:39.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:39.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:36:39.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:36:39.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:36:39.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:39.194 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[29586]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-06-14T22:36:39.195 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: Detected new or changed devices on smithi178 2024-06-14T22:36:39.195 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:39.195 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:39.195 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:36:39.195 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:36:39.195 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:36:39.195 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: Adjusting osd_memory_target on smithi178 to 5696M 2024-06-14T22:36:39.195 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:39.195 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:39.195 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:36:39.196 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:39.196 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:36:39.196 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:39.196 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:36:39.196 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:39.196 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:36:39.196 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:36:39.196 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:36:39.196 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:39.196 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:38 smithi046 ceph-mon[38286]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-06-14T22:36:39.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: Detected new or changed devices on smithi178 2024-06-14T22:36:39.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:39.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:39.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:36:39.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:36:39.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:36:39.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: Adjusting osd_memory_target on smithi178 to 5696M 2024-06-14T22:36:39.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:39.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:39.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:36:39.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:39.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:36:39.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:39.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:36:39.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:39.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: from='client.24289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:36:39.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-14T22:36:39.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-14T22:36:39.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:39.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:38 smithi178 ceph-mon[32942]: pgmap v158: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-06-14T22:36:41.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:41 smithi178 ceph-mon[32942]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-06-14T22:36:41.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:41 smithi178 ceph-mon[32942]: from='client.? 172.21.15.178:0/1475776637' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "205c8b05-359d-4ff3-a158-3e80e1c5fda6"}]: dispatch 2024-06-14T22:36:41.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:41 smithi178 ceph-mon[32942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "205c8b05-359d-4ff3-a158-3e80e1c5fda6"}]: dispatch 2024-06-14T22:36:41.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:41 smithi178 ceph-mon[32942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "205c8b05-359d-4ff3-a158-3e80e1c5fda6"}]': finished 2024-06-14T22:36:41.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:41 smithi178 ceph-mon[32942]: osdmap e52: 8 total, 7 up, 8 in 2024-06-14T22:36:41.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:41 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-14T22:36:41.526 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:41 smithi178 ceph-mon[32942]: from='client.? 172.21.15.178:0/2469494178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:36:41.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:41 smithi046 ceph-mon[29586]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-06-14T22:36:41.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:41 smithi046 ceph-mon[29586]: from='client.? 172.21.15.178:0/1475776637' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "205c8b05-359d-4ff3-a158-3e80e1c5fda6"}]: dispatch 2024-06-14T22:36:41.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:41 smithi046 ceph-mon[29586]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "205c8b05-359d-4ff3-a158-3e80e1c5fda6"}]: dispatch 2024-06-14T22:36:41.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:41 smithi046 ceph-mon[29586]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "205c8b05-359d-4ff3-a158-3e80e1c5fda6"}]': finished 2024-06-14T22:36:41.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:41 smithi046 ceph-mon[29586]: osdmap e52: 8 total, 7 up, 8 in 2024-06-14T22:36:41.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:41 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-14T22:36:41.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:41 smithi046 ceph-mon[29586]: from='client.? 172.21.15.178:0/2469494178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:36:41.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:41 smithi046 ceph-mon[38286]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-06-14T22:36:41.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:41 smithi046 ceph-mon[38286]: from='client.? 172.21.15.178:0/1475776637' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "205c8b05-359d-4ff3-a158-3e80e1c5fda6"}]: dispatch 2024-06-14T22:36:41.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:41 smithi046 ceph-mon[38286]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "205c8b05-359d-4ff3-a158-3e80e1c5fda6"}]: dispatch 2024-06-14T22:36:41.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:41 smithi046 ceph-mon[38286]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "205c8b05-359d-4ff3-a158-3e80e1c5fda6"}]': finished 2024-06-14T22:36:41.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:41 smithi046 ceph-mon[38286]: osdmap e52: 8 total, 7 up, 8 in 2024-06-14T22:36:41.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:41 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-14T22:36:41.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:41 smithi046 ceph-mon[38286]: from='client.? 172.21.15.178:0/2469494178' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-14T22:36:43.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:43 smithi046 ceph-mon[29586]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-06-14T22:36:43.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:43 smithi046 ceph-mon[38286]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-06-14T22:36:43.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:43 smithi178 ceph-mon[32942]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-06-14T22:36:45.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:45 smithi046 ceph-mon[29586]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-14T22:36:45.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:45 smithi046 ceph-mon[38286]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-14T22:36:45.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:45 smithi178 ceph-mon[32942]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-14T22:36:47.645 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:47 smithi178 ceph-mon[32942]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-06-14T22:36:47.645 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:47 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-14T22:36:47.645 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:47 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:47.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:47 smithi046 ceph-mon[29586]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-06-14T22:36:47.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:47 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-14T22:36:47.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:47 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:47.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:47 smithi046 ceph-mon[38286]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-06-14T22:36:47.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:47 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-14T22:36:47.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:47 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:36:48.693 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:48 smithi178 ceph-mon[32942]: Deploying daemon osd.7 on smithi178 2024-06-14T22:36:48.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:48 smithi046 ceph-mon[29586]: Deploying daemon osd.7 on smithi178 2024-06-14T22:36:48.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:48 smithi046 ceph-mon[38286]: Deploying daemon osd.7 on smithi178 2024-06-14T22:36:49.596 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:49 smithi178 ceph-mon[32942]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-14T22:36:49.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:49 smithi046 ceph-mon[29586]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-14T22:36:49.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:49 smithi046 ceph-mon[38286]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-14T22:36:51.521 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:51 smithi178 ceph-mon[32942]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-06-14T22:36:51.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:51 smithi046 ceph-mon[29586]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-06-14T22:36:51.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:51 smithi046 ceph-mon[38286]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-06-14T22:36:52.716 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:52 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:36:52.717 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:52 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:52.717 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:52 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:52.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:52 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:36:52.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:52 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:52.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:52 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:52.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:52 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:36:52.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:52 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:52.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:52 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:53.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:53 smithi178 ceph-mon[32942]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-06-14T22:36:53.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:53 smithi046 ceph-mon[29586]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-06-14T22:36:53.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:53 smithi046 ceph-mon[38286]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-06-14T22:36:55.364 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 7 on host 'smithi178' 2024-06-14T22:36:55.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:55 smithi178 ceph-mon[32942]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-06-14T22:36:55.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:55 smithi178 ceph-mon[32942]: from='osd.7 [v2:172.21.15.178:6824/826597267,v1:172.21.15.178:6825/826597267]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-14T22:36:55.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:55 smithi178 ceph-mon[32942]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-14T22:36:55.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:55 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:55.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:55 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:55.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:55 smithi046 ceph-mon[29586]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-06-14T22:36:55.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:55 smithi046 ceph-mon[29586]: from='osd.7 [v2:172.21.15.178:6824/826597267,v1:172.21.15.178:6825/826597267]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-14T22:36:55.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:55 smithi046 ceph-mon[29586]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-14T22:36:55.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:55 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:55.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:55 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:55.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:55 smithi046 ceph-mon[38286]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-06-14T22:36:55.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:55 smithi046 ceph-mon[38286]: from='osd.7 [v2:172.21.15.178:6824/826597267,v1:172.21.15.178:6825/826597267]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-14T22:36:55.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:55 smithi046 ceph-mon[38286]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-14T22:36:55.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:55 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:55.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:55 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:56.132 DEBUG:teuthology.orchestra.run.smithi178:osd.7> sudo journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.7.service 2024-06-14T22:36:56.134 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-06-14T22:36:56.134 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd stat -f json 2024-06-14T22:36:56.414 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:36:56.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:36:56 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:36:56.467+0000 7f243497a640 -1 osd.7 0 waiting for initial osdmap 2024-06-14T22:36:56.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:36:56 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:36:56.475+0000 7f242f7b1640 -1 osd.7 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-14T22:36:56.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:56 smithi178 ceph-mon[32942]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-14T22:36:56.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:56 smithi178 ceph-mon[32942]: osdmap e53: 8 total, 7 up, 8 in 2024-06-14T22:36:56.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:56 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-14T22:36:56.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:56 smithi178 ceph-mon[32942]: from='osd.7 [v2:172.21.15.178:6824/826597267,v1:172.21.15.178:6825/826597267]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:36:56.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:56 smithi178 ceph-mon[32942]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:36:56.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:56 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:56.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:56 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:56.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:56 smithi046 ceph-mon[29586]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-14T22:36:56.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:56 smithi046 ceph-mon[29586]: osdmap e53: 8 total, 7 up, 8 in 2024-06-14T22:36:56.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:56 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-14T22:36:56.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:56 smithi046 ceph-mon[29586]: from='osd.7 [v2:172.21.15.178:6824/826597267,v1:172.21.15.178:6825/826597267]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:36:56.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:56 smithi046 ceph-mon[29586]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:36:56.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:56 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:56.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:56 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:56.777 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:56 smithi046 ceph-mon[38286]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-14T22:36:56.777 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:56 smithi046 ceph-mon[38286]: osdmap e53: 8 total, 7 up, 8 in 2024-06-14T22:36:56.777 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:56 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-14T22:36:56.777 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:56 smithi046 ceph-mon[38286]: from='osd.7 [v2:172.21.15.178:6824/826597267,v1:172.21.15.178:6825/826597267]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:36:56.777 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:56 smithi046 ceph-mon[38286]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-06-14T22:36:56.777 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:56 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:56.777 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:56 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:36:57.637 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:57 smithi046 ceph-mon[38286]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-06-14T22:36:57.637 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:57 smithi046 ceph-mon[38286]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-06-14T22:36:57.637 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:57 smithi046 ceph-mon[38286]: osdmap e54: 8 total, 7 up, 8 in 2024-06-14T22:36:57.637 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:57 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-14T22:36:57.638 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:57 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-14T22:36:57.638 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:57 smithi046 ceph-mon[29586]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-06-14T22:36:57.638 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:57 smithi046 ceph-mon[29586]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-06-14T22:36:57.638 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:57 smithi046 ceph-mon[29586]: osdmap e54: 8 total, 7 up, 8 in 2024-06-14T22:36:57.638 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:57 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-14T22:36:57.638 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:57 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-14T22:36:57.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:57 smithi178 ceph-mon[32942]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-06-14T22:36:57.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:57 smithi178 ceph-mon[32942]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-06-14T22:36:57.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:57 smithi178 ceph-mon[32942]: osdmap e54: 8 total, 7 up, 8 in 2024-06-14T22:36:57.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:57 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-14T22:36:57.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:57 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-14T22:36:58.338 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:36:58.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:58 smithi046 ceph-mon[29586]: purged_snaps scrub starts 2024-06-14T22:36:58.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:58 smithi046 ceph-mon[29586]: purged_snaps scrub ok 2024-06-14T22:36:58.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:58 smithi046 ceph-mon[29586]: osd.7 [v2:172.21.15.178:6824/826597267,v1:172.21.15.178:6825/826597267] boot 2024-06-14T22:36:58.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:58 smithi046 ceph-mon[29586]: osdmap e55: 8 total, 8 up, 8 in 2024-06-14T22:36:58.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:58 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-14T22:36:58.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:58 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/3214710807' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-14T22:36:58.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:58 smithi046 ceph-mon[29586]: osdmap e56: 8 total, 8 up, 8 in 2024-06-14T22:36:58.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:58 smithi046 ceph-mon[38286]: purged_snaps scrub starts 2024-06-14T22:36:58.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:58 smithi046 ceph-mon[38286]: purged_snaps scrub ok 2024-06-14T22:36:58.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:58 smithi046 ceph-mon[38286]: osd.7 [v2:172.21.15.178:6824/826597267,v1:172.21.15.178:6825/826597267] boot 2024-06-14T22:36:58.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:58 smithi046 ceph-mon[38286]: osdmap e55: 8 total, 8 up, 8 in 2024-06-14T22:36:58.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:58 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-14T22:36:58.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:58 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/3214710807' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-14T22:36:58.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:58 smithi046 ceph-mon[38286]: osdmap e56: 8 total, 8 up, 8 in 2024-06-14T22:36:58.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:58 smithi178 ceph-mon[32942]: purged_snaps scrub starts 2024-06-14T22:36:58.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:58 smithi178 ceph-mon[32942]: purged_snaps scrub ok 2024-06-14T22:36:58.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:58 smithi178 ceph-mon[32942]: osd.7 [v2:172.21.15.178:6824/826597267,v1:172.21.15.178:6825/826597267] boot 2024-06-14T22:36:58.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:58 smithi178 ceph-mon[32942]: osdmap e55: 8 total, 8 up, 8 in 2024-06-14T22:36:58.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:58 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-14T22:36:58.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:58 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/3214710807' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-14T22:36:58.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:58 smithi178 ceph-mon[32942]: osdmap e56: 8 total, 8 up, 8 in 2024-06-14T22:36:59.081 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":55,"num_osds":8,"num_up_osds":8,"osd_up_since":1718404617,"num_in_osds":8,"osd_in_since":1718404600,"num_remapped_pgs":1} 2024-06-14T22:36:59.082 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd dump --format=json 2024-06-14T22:36:59.345 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:36:59.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:36:59 smithi178 ceph-mon[32942]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:36:59.731 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:36:59 smithi046 ceph-mon[29586]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:36:59.731 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:36:59 smithi046 ceph-mon[38286]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:00.770 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[38286]: osdmap e57: 8 total, 8 up, 8 in 2024-06-14T22:37:00.770 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[38286]: Detected new or changed devices on smithi178 2024-06-14T22:37:00.770 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:00.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:00.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:00.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:00.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:00.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:00.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[38286]: Adjusting osd_memory_target on smithi178 to 4272M 2024-06-14T22:37:00.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:00.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:37:00.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:37:00.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:00.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:37:00.772 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:37:00.772 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:37:00.772 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:00.772 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[29586]: osdmap e57: 8 total, 8 up, 8 in 2024-06-14T22:37:00.772 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[29586]: Detected new or changed devices on smithi178 2024-06-14T22:37:00.772 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:00.772 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:00.772 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:00.772 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:00.772 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:00.772 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:00.773 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[29586]: Adjusting osd_memory_target on smithi178 to 4272M 2024-06-14T22:37:00.773 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:00.773 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:37:00.773 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:37:00.773 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:00.773 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:37:00.773 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:37:00.773 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:37:00.773 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:00 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:00.778 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:00 smithi178 ceph-mon[32942]: osdmap e57: 8 total, 8 up, 8 in 2024-06-14T22:37:00.778 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:00 smithi178 ceph-mon[32942]: Detected new or changed devices on smithi178 2024-06-14T22:37:00.778 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:00 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:00.778 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:00 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:00.779 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:00 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:00.779 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:00 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:00.779 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:00 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:00.779 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:00 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:00.779 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:00 smithi178 ceph-mon[32942]: Adjusting osd_memory_target on smithi178 to 4272M 2024-06-14T22:37:00.779 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:00 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:00.779 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:00 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:37:00.779 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:00 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:37:00.779 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:00 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:00.780 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:00 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:37:00.780 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:00 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:37:00.780 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:00 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:37:00.780 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:00 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:01.222 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:37:01.222 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":58,"fsid":"cb009720-2a9d-11ef-bc9d-c7b262605968","created":"2024-06-14T22:31:50.709367+0000","modified":"2024-06-14T22:37:00.472822+0000","last_up_change":"2024-06-14T22:36:57.461962+0000","last_in_change":"2024-06-14T22:36:40.661606+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-14T22:34:58.841776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"33945598-f864-40e6-904e-1bff72ec8b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6803","nonce":2467893990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6805","nonce":2467893990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6809","nonce":2467893990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6807","nonce":2467893990}]},"public_addr":"172.21.15.46:6803/2467893990","cluster_addr":"172.21.15.46:6805/2467893990","heartbeat_back_addr":"172.21.15.46:6809/2467893990","heartbeat_front_addr":"172.21.15.46:6807/2467893990","state":["exists","up"]},{"osd":1,"uuid":"48f0e5ef-2d0a-4858-a629-69f82c604f9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6811","nonce":2133775041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6813","nonce":2133775041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6817","nonce":2133775041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6815","nonce":2133775041}]},"public_addr":"172.21.15.46:6811/2133775041","cluster_addr":"172.21.15.46:6813/2133775041","heartbeat_back_addr":"172.21.15.46:6817/2133775041","heartbeat_front_addr":"172.21.15.46:6815/2133775041","state":["exists","up"]},{"osd":2,"uuid":"74b998f5-fb99-4c37-9463-11dd602f1364","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6819","nonce":3036855066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6821","nonce":3036855066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6825","nonce":3036855066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6823","nonce":3036855066}]},"public_addr":"172.21.15.46:6819/3036855066","cluster_addr":"172.21.15.46:6821/3036855066","heartbeat_back_addr":"172.21.15.46:6825/3036855066","heartbeat_front_addr":"172.21.15.46:6823/3036855066","state":["exists","up"]},{"osd":3,"uuid":"e4bd323c-2885-47a0-a44c-275077fc4b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6827","nonce":2597052465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6829","nonce":2597052465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6833","nonce":2597052465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6831","nonce":2597052465}]},"public_addr":"172.21.15.46:6827/2597052465","cluster_addr":"172.21.15.46:6829/2597052465","heartbeat_back_addr":"172.21.15.46:6833/2597052465","heartbeat_front_addr":"172.21.15.46:6831/2597052465","state":["exists","up"]},{"osd":4,"uuid":"0a0b2d3d-3e61-47be-a95f-8294f79ecc7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6801","nonce":2179289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6803","nonce":2179289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6807","nonce":2179289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6805","nonce":2179289524}]},"public_addr":"172.21.15.178:6801/2179289524","cluster_addr":"172.21.15.178:6803/2179289524","heartbeat_back_addr":"172.21.15.178:6807/2179289524","heartbeat_front_addr":"172.21.15.178:6805/2179289524","state":["exists","up"]},{"osd":5,"uuid":"29f79df4-039f-44a3-8570-7252eca3abe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6809","nonce":2813027226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6811","nonce":2813027226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6815","nonce":2813027226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6813","nonce":2813027226}]},"public_addr":"172.21.15.178:6809/2813027226","cluster_addr":"172.21.15.178:6811/2813027226","heartbeat_back_addr":"172.21.15.178:6815/2813027226","heartbeat_front_addr":"172.21.15.178:6813/2813027226","state":["exists","up"]},{"osd":6,"uuid":"56661a2b-4e13-47e7-972a-bb1861d5e33d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6817","nonce":3396977870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6819","nonce":3396977870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6823","nonce":3396977870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6821","nonce":3396977870}]},"public_addr":"172.21.15.178:6817/3396977870","cluster_addr":"172.21.15.178:6819/3396977870","heartbeat_back_addr":"172.21.15.178:6823/3396977870","heartbeat_front_addr":"172.21.15.178:6821/3396977870","state":["exists","up"]},{"osd":7,"uuid":"205c8b05-359d-4ff3-a158-3e80e1c5fda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6825","nonce":826597267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6827","nonce":826597267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6831","nonce":826597267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6829","nonce":826597267}]},"public_addr":"172.21.15.178:6825/826597267","cluster_addr":"172.21.15.178:6827/826597267","heartbeat_back_addr":"172.21.15.178:6831/826597267","heartbeat_front_addr":"172.21.15.178:6829/826597267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:34:06.575453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:34:31.020820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:34:55.617857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:35:20.862506+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:35:42.941353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:36:06.653807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:36:31.539489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:36:56.167994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/4238739813":"2024-06-15T22:32:44.342121+0000","172.21.15.46:6800/4238739813":"2024-06-15T22:32:44.342121+0000","172.21.15.46:0/96950257":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/3550067878":"2024-06-15T22:32:44.342121+0000","172.21.15.46:0/1630559323":"2024-06-15T22:32:44.342121+0000","172.21.15.46:0/4205563548":"2024-06-15T22:32:22.225646+0000","172.21.15.46:6801/1612763800":"2024-06-15T22:32:22.225646+0000","172.21.15.46:6800/1612763800":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/2833589127":"2024-06-15T22:32:44.342121+0000","172.21.15.46:0/1856025406":"2024-06-15T22:32:22.225646+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-14T22:37:01.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:01 smithi178 ceph-mon[32942]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:01.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:01 smithi178 ceph-mon[32942]: osdmap e58: 8 total, 8 up, 8 in 2024-06-14T22:37:01.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:01 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/1676107486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-14T22:37:01.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:01 smithi178 ceph-mon[32942]: osdmap e59: 8 total, 8 up, 8 in 2024-06-14T22:37:01.901 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:01 smithi046 ceph-mon[38286]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:01.901 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:01 smithi046 ceph-mon[38286]: osdmap e58: 8 total, 8 up, 8 in 2024-06-14T22:37:01.901 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:01 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/1676107486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-14T22:37:01.901 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:01 smithi046 ceph-mon[38286]: osdmap e59: 8 total, 8 up, 8 in 2024-06-14T22:37:01.901 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:01 smithi046 ceph-mon[29586]: pgmap v175: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:01.901 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:01 smithi046 ceph-mon[29586]: osdmap e58: 8 total, 8 up, 8 in 2024-06-14T22:37:01.902 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:01 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/1676107486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-14T22:37:01.902 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:01 smithi046 ceph-mon[29586]: osdmap e59: 8 total, 8 up, 8 in 2024-06-14T22:37:01.933 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-14T22:34:58.841776+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-06-14T22:37:01.933 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-14T22:37:02.203 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:37:03.719 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:03 smithi046 ceph-mon[29586]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:03.720 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:03 smithi046 ceph-mon[38286]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:03.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:03 smithi178 ceph-mon[32942]: pgmap v178: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:03.990 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-06-14T22:37:04.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:04 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/4269450533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-14T22:37:04.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:04 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/4269450533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-14T22:37:04.761 INFO:tasks.cephadm:Adding prometheus.a on smithi178 2024-06-14T22:37:04.761 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph orch apply prometheus '1;smithi178=a' 2024-06-14T22:37:04.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:04 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/4269450533' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-14T22:37:05.023 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.b/config 2024-06-14T22:37:05.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:05 smithi178 ceph-mon[32942]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-06-14T22:37:05.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:05 smithi046 ceph-mon[29586]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-06-14T22:37:05.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:05 smithi046 ceph-mon[38286]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-06-14T22:37:06.732 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled prometheus update... 2024-06-14T22:37:07.418 DEBUG:teuthology.orchestra.run.smithi178:prometheus.a> sudo journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@prometheus.a.service 2024-06-14T22:37:07.462 INFO:tasks.cephadm:Adding node-exporter.a on smithi046 2024-06-14T22:37:07.462 INFO:tasks.cephadm:Adding node-exporter.b on smithi178 2024-06-14T22:37:07.462 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph orch apply node-exporter '2;smithi046=a;smithi178=b' 2024-06-14T22:37:07.756 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.b/config 2024-06-14T22:37:07.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:07 smithi178 ceph-mon[32942]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-06-14T22:37:07.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:07 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:07.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:07 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:37:07.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:07 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:37:07.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:07 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:37:07.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:07 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:07.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:07 smithi046 ceph-mon[29586]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-06-14T22:37:07.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:07 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:07.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:07 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:37:07.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:07 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:37:07.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:07 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:37:07.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:07 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:07.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:07 smithi046 ceph-mon[38286]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-06-14T22:37:07.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:07 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:07.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:07 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:37:07.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:07 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:37:07.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:07 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:37:07.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:07 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:08.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:08 smithi178 ceph-mon[32942]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi178=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:37:08.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:08 smithi178 ceph-mon[32942]: Saving service prometheus spec with placement smithi178=a;count:1 2024-06-14T22:37:08.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:08 smithi178 ceph-mon[32942]: Deploying daemon prometheus.a on smithi178 2024-06-14T22:37:08.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:08 smithi046 ceph-mon[29586]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi178=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:37:08.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:08 smithi046 ceph-mon[29586]: Saving service prometheus spec with placement smithi178=a;count:1 2024-06-14T22:37:08.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:08 smithi046 ceph-mon[29586]: Deploying daemon prometheus.a on smithi178 2024-06-14T22:37:08.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:08 smithi046 ceph-mon[38286]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi178=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:37:08.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:08 smithi046 ceph-mon[38286]: Saving service prometheus spec with placement smithi178=a;count:1 2024-06-14T22:37:08.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:08 smithi046 ceph-mon[38286]: Deploying daemon prometheus.a on smithi178 2024-06-14T22:37:09.708 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:09 smithi178 ceph-mon[32942]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-14T22:37:09.942 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:09 smithi046 ceph-mon[29586]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-14T22:37:09.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:09 smithi046 ceph-mon[38286]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-14T22:37:10.144 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled node-exporter update... 2024-06-14T22:37:11.106 DEBUG:teuthology.orchestra.run.smithi046:node-exporter.a> sudo journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@node-exporter.a.service 2024-06-14T22:37:11.109 DEBUG:teuthology.orchestra.run.smithi178:node-exporter.b> sudo journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@node-exporter.b.service 2024-06-14T22:37:11.112 INFO:tasks.cephadm:Adding alertmanager.a on smithi046 2024-06-14T22:37:11.112 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph orch apply alertmanager '1;smithi046=a' 2024-06-14T22:37:11.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:11 smithi046 ceph-mon[29586]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi046=a;smithi178=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:37:11.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:11 smithi046 ceph-mon[29586]: Saving service node-exporter spec with placement smithi046=a;smithi178=b;count:2 2024-06-14T22:37:11.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:11 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:11.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:11 smithi046 ceph-mon[29586]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-14T22:37:11.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:11 smithi046 ceph-mon[38286]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi046=a;smithi178=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:37:11.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:11 smithi046 ceph-mon[38286]: Saving service node-exporter spec with placement smithi046=a;smithi178=b;count:2 2024-06-14T22:37:11.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:11 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:11.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:11 smithi046 ceph-mon[38286]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-14T22:37:11.466 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.b/config 2024-06-14T22:37:11.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:11 smithi178 ceph-mon[32942]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi046=a;smithi178=b", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:37:11.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:11 smithi178 ceph-mon[32942]: Saving service node-exporter spec with placement smithi046=a;smithi178=b;count:2 2024-06-14T22:37:11.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:11 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:11.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:11 smithi178 ceph-mon[32942]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-14T22:37:13.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:13 smithi046 ceph-mon[29586]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-06-14T22:37:13.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:13 smithi046 ceph-mon[38286]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-06-14T22:37:13.720 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:13 smithi178 ceph-mon[32942]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-06-14T22:37:14.007 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled alertmanager update... 2024-06-14T22:37:15.269 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:15 smithi178 ceph-mon[32942]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi046=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:37:15.269 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:15 smithi178 ceph-mon[32942]: Saving service alertmanager spec with placement smithi046=a;count:1 2024-06-14T22:37:15.269 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:15 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:15.269 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:15 smithi178 ceph-mon[32942]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-14T22:37:15.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:15 smithi046 ceph-mon[29586]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi046=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:37:15.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:15 smithi046 ceph-mon[29586]: Saving service alertmanager spec with placement smithi046=a;count:1 2024-06-14T22:37:15.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:15 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:15.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:15 smithi046 ceph-mon[29586]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-14T22:37:15.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:15 smithi046 ceph-mon[38286]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi046=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:37:15.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:15 smithi046 ceph-mon[38286]: Saving service alertmanager spec with placement smithi046=a;count:1 2024-06-14T22:37:15.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:15 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:15.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:15 smithi046 ceph-mon[38286]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-14T22:37:15.847 DEBUG:teuthology.orchestra.run.smithi046:alertmanager.a> sudo journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@alertmanager.a.service 2024-06-14T22:37:15.850 INFO:tasks.cephadm:Adding grafana.a on smithi178 2024-06-14T22:37:15.851 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph orch apply grafana '1;smithi178=a' 2024-06-14T22:37:16.393 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.b/config 2024-06-14T22:37:17.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:17 smithi178 ceph-mon[32942]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:17.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:17 smithi046 ceph-mon[29586]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:17.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:17 smithi046 ceph-mon[38286]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:19.193 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled grafana update... 2024-06-14T22:37:19.496 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:19 smithi178 ceph-mon[32942]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:19.496 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:19 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:19.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:19 smithi046 ceph-mon[29586]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:19.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:19 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:19.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:19 smithi046 ceph-mon[38286]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:19.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:19 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:19.797 DEBUG:teuthology.orchestra.run.smithi178:grafana.a> sudo journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@grafana.a.service 2024-06-14T22:37:19.800 INFO:tasks.cephadm:Setting up client nodes... 2024-06-14T22:37:19.800 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-14T22:37:20.073 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:37:20.452 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:20 smithi046 ceph-mon[29586]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi178=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:37:20.452 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:20 smithi046 ceph-mon[29586]: Saving service grafana spec with placement smithi178=a;count:1 2024-06-14T22:37:20.452 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:20 smithi046 ceph-mon[38286]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi178=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:37:20.452 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:20 smithi046 ceph-mon[38286]: Saving service grafana spec with placement smithi178=a;count:1 2024-06-14T22:37:20.710 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:20 smithi178 ceph-mon[32942]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi178=a", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:37:20.711 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:20 smithi178 ceph-mon[32942]: Saving service grafana spec with placement smithi178=a;count:1 2024-06-14T22:37:21.633 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:21 smithi178 ceph-mon[32942]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:21.910 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-06-14T22:37:21.910 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQAhxmxma2dPNhAAvvRFopKT6H+CMoSHB8B6wQ== 2024-06-14T22:37:21.911 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:21 smithi046 ceph-mon[29586]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:21.911 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:21 smithi046 ceph-mon[38286]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:22.275 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:22 smithi178 systemd[1]: Starting Ceph prometheus.a for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:37:22.564 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-06-14T22:37:22.564 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-14T22:37:22.564 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-14T22:37:22.606 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-14T22:37:22.631 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:22 smithi178 podman[61572]: 2024-06-14 22:37:22.285775659 +0000 UTC m=+0.018248040 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-06-14T22:37:22.632 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:22 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/1485652911' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-14T22:37:22.632 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:22 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/1485652911' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-14T22:37:22.766 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:22 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/1485652911' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-14T22:37:22.766 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:22 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/1485652911' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-14T22:37:22.767 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:22 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/1485652911' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-14T22:37:22.767 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:22 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/1485652911' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-14T22:37:23.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:23 smithi178 ceph-mon[32942]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:23.775 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:23 smithi178 podman[61572]: 2024-06-14 22:37:23.434993035 +0000 UTC m=+1.167465401 container create 5ebccc451606087474e4b8e9b46cd83cefd32f5c3a1a61e2371e334f80d66a3a (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-14T22:37:23.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:23 smithi046 ceph-mon[29586]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:23.943 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:23 smithi046 ceph-mon[38286]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:24.363 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.b/config 2024-06-14T22:37:24.493 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 podman[61572]: 2024-06-14 22:37:24.246214994 +0000 UTC m=+1.978687378 container init 5ebccc451606087474e4b8e9b46cd83cefd32f5c3a1a61e2371e334f80d66a3a (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-14T22:37:24.494 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 podman[61572]: 2024-06-14 22:37:24.253052053 +0000 UTC m=+1.985524427 container start 5ebccc451606087474e4b8e9b46cd83cefd32f5c3a1a61e2371e334f80d66a3a (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-14T22:37:24.494 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.279Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-06-14T22:37:24.494 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.279Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-06-14T22:37:24.494 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.279Z caller=main.go:570 level=info host_details="(Linux 5.14.0-457.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu May 30 22:48:58 UTC 2024 x86_64 smithi178 (none))" 2024-06-14T22:37:24.494 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.279Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-06-14T22:37:24.494 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.279Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-06-14T22:37:24.494 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.280Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-06-14T22:37:24.494 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.281Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-06-14T22:37:24.494 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.282Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-06-14T22:37:24.494 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.282Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-06-14T22:37:24.495 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.285Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-06-14T22:37:24.495 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.285Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.317µs 2024-06-14T22:37:24.495 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.285Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-06-14T22:37:24.495 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.285Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-06-14T22:37:24.495 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.285Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=18.955µs wal_replay_duration=281.409µs wbl_replay_duration=149ns total_replay_duration=319.658µs 2024-06-14T22:37:24.495 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.286Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-06-14T22:37:24.495 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.286Z caller=main.go:1029 level=info msg="TSDB started" 2024-06-14T22:37:24.495 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.286Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-06-14T22:37:24.495 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.300Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.279155ms db_storage=841ns remote_storage=1.234µs web_handler=418ns query_engine=610ns scrape=469.893µs scrape_sd=18.622µs notify=730ns notify_sd=1.908µs rules=13.605059ms tracing=6.221µs 2024-06-14T22:37:24.495 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.300Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-06-14T22:37:24.495 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:37:24.300Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-06-14T22:37:24.495 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 bash[61572]: 5ebccc451606087474e4b8e9b46cd83cefd32f5c3a1a61e2371e334f80d66a3a 2024-06-14T22:37:24.495 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:37:24 smithi178 systemd[1]: Started Ceph prometheus.a for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:37:25.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:25 smithi178 ceph-mon[32942]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:25.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:25 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:25.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:25 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:25.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:25 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:25.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:25 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-06-14T22:37:25.525 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:25 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: ignoring --setuser ceph since I am not root 2024-06-14T22:37:25.525 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:25 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: ignoring --setgroup ceph since I am not root 2024-06-14T22:37:25.526 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:25 smithi178 ceph-mgr[35018]: -- 172.21.15.178:0/818622312 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5589d6b3a9c0 con 0x5589d6b47000 2024-06-14T22:37:25.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:25 smithi046 ceph-mon[38286]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:25.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:25 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:25.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:25 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:25.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:25 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:25.692 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:25 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-06-14T22:37:25.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:25 smithi046 ceph-mon[29586]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:25.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:25 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:25.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:25 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:25.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:25 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' 2024-06-14T22:37:25.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:25 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-06-14T22:37:25.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:25 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ignoring --setuser ceph since I am not root 2024-06-14T22:37:25.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:25 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ignoring --setgroup ceph since I am not root 2024-06-14T22:37:25.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:25 smithi046 ceph-mgr[29866]: -- 172.21.15.46:0/4044729730 <== mon.2 v2:172.21.15.46:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55e64e36c9c0 con 0x55e64e378800 2024-06-14T22:37:25.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:25 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:25.563+0000 7f8d10873ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-14T22:37:25.694 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:25 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:25.694+0000 7f8d10873ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-14T22:37:25.875 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:25 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:25.557+0000 7f726b294ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-14T22:37:25.876 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:25 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:25.690+0000 7f726b294ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-14T22:37:25.876 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:25 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:25.757+0000 7f726b294ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-14T22:37:25.942 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:25 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:25.764+0000 7f8d10873ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-14T22:37:26.275 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:26 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:26.010+0000 7f726b294ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-14T22:37:26.275 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:26 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:26.071+0000 7f726b294ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-14T22:37:26.335 INFO:teuthology.orchestra.run.smithi178.stdout:[client.1] 2024-06-14T22:37:26.335 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQAmxmxmuQEGFBAA1uLz3WlyAL2NhYEKylQpbw== 2024-06-14T22:37:26.396 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:26.020+0000 7f8d10873ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-14T22:37:26.396 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:26.082+0000 7f8d10873ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-14T22:37:26.396 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:26.399+0000 7f8d10873ec0 -1 mgr[py] Module not found: 'k8sevents' 2024-06-14T22:37:26.624 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:26 smithi178 ceph-mon[32942]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-06-14T22:37:26.624 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:26 smithi178 ceph-mon[32942]: mgrmap e16: y(active, since 4m), standbys: x 2024-06-14T22:37:26.624 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:26 smithi178 ceph-mon[32942]: from='client.? 172.21.15.178:0/85550090' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-14T22:37:26.624 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:26 smithi178 ceph-mon[32942]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-14T22:37:26.624 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:26 smithi178 ceph-mon[32942]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-14T22:37:26.625 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:26 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:26.390+0000 7f726b294ec0 -1 mgr[py] Module not found: 'k8sevents' 2024-06-14T22:37:26.625 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:26 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:26.390+0000 7f726b294ec0 -1 mgr[py] Traceback (most recent call last): 2024-06-14T22:37:26.625 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:26 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: File "/usr/share/ceph/mgr/k8sevents/__init__.py", line 1, in 2024-06-14T22:37:26.625 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:26 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: from .module import Module 2024-06-14T22:37:26.625 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:26 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: File "/usr/share/ceph/mgr/k8sevents/module.py", line 70, in 2024-06-14T22:37:26.625 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:26 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: from kubernetes.client.models.v1_event import V1Event 2024-06-14T22:37:26.625 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:26 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: ModuleNotFoundError: No module named 'kubernetes.client.models.v1_event' 2024-06-14T22:37:26.625 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:26 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:26.625 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:26 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:26.390+0000 7f726b294ec0 -1 mgr[py] Class not found in module 'k8sevents' 2024-06-14T22:37:26.625 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:26 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:26.390+0000 7f726b294ec0 -1 mgr[py] Error loading module 'k8sevents': (2) No such file or directory 2024-06-14T22:37:26.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-mon[29586]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-06-14T22:37:26.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-mon[29586]: mgrmap e16: y(active, since 4m), standbys: x 2024-06-14T22:37:26.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-mon[29586]: from='client.? 172.21.15.178:0/85550090' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-14T22:37:26.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-mon[29586]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-14T22:37:26.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-mon[29586]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-14T22:37:26.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-mon[38286]: from='mgr.14150 172.21.15.46:0/2607087195' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-06-14T22:37:26.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-mon[38286]: mgrmap e16: y(active, since 4m), standbys: x 2024-06-14T22:37:26.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-mon[38286]: from='client.? 172.21.15.178:0/85550090' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-14T22:37:26.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-mon[38286]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-14T22:37:26.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-mon[38286]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-14T22:37:26.694 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:26.399+0000 7f8d10873ec0 -1 mgr[py] Traceback (most recent call last): 2024-06-14T22:37:26.694 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/usr/share/ceph/mgr/k8sevents/__init__.py", line 1, in 2024-06-14T22:37:26.694 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: from .module import Module 2024-06-14T22:37:26.694 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/usr/share/ceph/mgr/k8sevents/module.py", line 70, in 2024-06-14T22:37:26.694 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: from kubernetes.client.models.v1_event import V1Event 2024-06-14T22:37:26.694 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ModuleNotFoundError: No module named 'kubernetes.client.models.v1_event' 2024-06-14T22:37:26.694 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:26.695 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:26.400+0000 7f8d10873ec0 -1 mgr[py] Class not found in module 'k8sevents' 2024-06-14T22:37:26.695 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:26 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:26.400+0000 7f8d10873ec0 -1 mgr[py] Error loading module 'k8sevents': (2) No such file or directory 2024-06-14T22:37:27.115 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-06-14T22:37:27.115 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-06-14T22:37:27.115 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-06-14T22:37:27.194 INFO:tasks.cephadm:Waiting for OSDs to come up 2024-06-14T22:37:27.195 DEBUG:teuthology.orchestra.run.smithi046:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-06-14T22:37:27.425 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:27 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:27.036+0000 7f726b294ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-14T22:37:27.425 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:27 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:27.167+0000 7f726b294ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-14T22:37:27.426 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:27 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:27.227+0000 7f726b294ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-14T22:37:27.426 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:27 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:27.429+0000 7f726b294ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-14T22:37:27.441 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:27 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:27.042+0000 7f8d10873ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-14T22:37:27.442 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:27 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:27.160+0000 7f8d10873ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-14T22:37:27.442 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:27 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:27.220+0000 7f8d10873ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-14T22:37:27.442 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:27 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:27.445+0000 7f8d10873ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-14T22:37:27.509 INFO:teuthology.misc.health.smithi046.stdout: 2024-06-14T22:37:27.509 INFO:teuthology.misc.health.smithi046.stdout:{"epoch":59,"fsid":"cb009720-2a9d-11ef-bc9d-c7b262605968","created":"2024-06-14T22:31:50.709367+0000","modified":"2024-06-14T22:37:01.474390+0000","last_up_change":"2024-06-14T22:36:57.461962+0000","last_in_change":"2024-06-14T22:36:40.661606+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-14T22:34:58.841776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"33945598-f864-40e6-904e-1bff72ec8b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6803","nonce":2467893990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6805","nonce":2467893990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6809","nonce":2467893990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6807","nonce":2467893990}]},"public_addr":"172.21.15.46:6803/2467893990","cluster_addr":"172.21.15.46:6805/2467893990","heartbeat_back_addr":"172.21.15.46:6809/2467893990","heartbeat_front_addr":"172.21.15.46:6807/2467893990","state":["exists","up"]},{"osd":1,"uuid":"48f0e5ef-2d0a-4858-a629-69f82c604f9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6811","nonce":2133775041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6813","nonce":2133775041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6817","nonce":2133775041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6815","nonce":2133775041}]},"public_addr":"172.21.15.46:6811/2133775041","cluster_addr":"172.21.15.46:6813/2133775041","heartbeat_back_addr":"172.21.15.46:6817/2133775041","heartbeat_front_addr":"172.21.15.46:6815/2133775041","state":["exists","up"]},{"osd":2,"uuid":"74b998f5-fb99-4c37-9463-11dd602f1364","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6819","nonce":3036855066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6821","nonce":3036855066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6825","nonce":3036855066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6823","nonce":3036855066}]},"public_addr":"172.21.15.46:6819/3036855066","cluster_addr":"172.21.15.46:6821/3036855066","heartbeat_back_addr":"172.21.15.46:6825/3036855066","heartbeat_front_addr":"172.21.15.46:6823/3036855066","state":["exists","up"]},{"osd":3,"uuid":"e4bd323c-2885-47a0-a44c-275077fc4b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6827","nonce":2597052465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6829","nonce":2597052465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6833","nonce":2597052465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6831","nonce":2597052465}]},"public_addr":"172.21.15.46:6827/2597052465","cluster_addr":"172.21.15.46:6829/2597052465","heartbeat_back_addr":"172.21.15.46:6833/2597052465","heartbeat_front_addr":"172.21.15.46:6831/2597052465","state":["exists","up"]},{"osd":4,"uuid":"0a0b2d3d-3e61-47be-a95f-8294f79ecc7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6801","nonce":2179289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6803","nonce":2179289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6807","nonce":2179289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6805","nonce":2179289524}]},"public_addr":"172.21.15.178:6801/2179289524","cluster_addr":"172.21.15.178:6803/2179289524","heartbeat_back_addr":"172.21.15.178:6807/2179289524","heartbeat_front_addr":"172.21.15.178:6805/2179289524","state":["exists","up"]},{"osd":5,"uuid":"29f79df4-039f-44a3-8570-7252eca3abe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6809","nonce":2813027226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6811","nonce":2813027226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6815","nonce":2813027226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6813","nonce":2813027226}]},"public_addr":"172.21.15.178:6809/2813027226","cluster_addr":"172.21.15.178:6811/2813027226","heartbeat_back_addr":"172.21.15.178:6815/2813027226","heartbeat_front_addr":"172.21.15.178:6813/2813027226","state":["exists","up"]},{"osd":6,"uuid":"56661a2b-4e13-47e7-972a-bb1861d5e33d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6817","nonce":3396977870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6819","nonce":3396977870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6823","nonce":3396977870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6821","nonce":3396977870}]},"public_addr":"172.21.15.178:6817/3396977870","cluster_addr":"172.21.15.178:6819/3396977870","heartbeat_back_addr":"172.21.15.178:6823/3396977870","heartbeat_front_addr":"172.21.15.178:6821/3396977870","state":["exists","up"]},{"osd":7,"uuid":"205c8b05-359d-4ff3-a158-3e80e1c5fda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6825","nonce":826597267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6827","nonce":826597267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6831","nonce":826597267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6829","nonce":826597267}]},"public_addr":"172.21.15.178:6825/826597267","cluster_addr":"172.21.15.178:6827/826597267","heartbeat_back_addr":"172.21.15.178:6831/826597267","heartbeat_front_addr":"172.21.15.178:6829/826597267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:34:06.575453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:34:31.020820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:34:55.617857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:35:20.862506+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:35:42.941353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:36:06.653807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:36:31.539489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:36:56.167994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6801/4238739813":"2024-06-15T22:32:44.342121+0000","172.21.15.46:6800/4238739813":"2024-06-15T22:32:44.342121+0000","172.21.15.46:0/96950257":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/3550067878":"2024-06-15T22:32:44.342121+0000","172.21.15.46:0/1630559323":"2024-06-15T22:32:44.342121+0000","172.21.15.46:0/4205563548":"2024-06-15T22:32:22.225646+0000","172.21.15.46:6801/1612763800":"2024-06-15T22:32:22.225646+0000","172.21.15.46:6800/1612763800":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/2833589127":"2024-06-15T22:32:44.342121+0000","172.21.15.46:0/1856025406":"2024-06-15T22:32:22.225646+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-14T22:37:27.520 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-06-14T22:37:27.521 INFO:tasks.cephadm:Creating RBD pool 2024-06-14T22:37:27.521 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- sudo ceph --cluster ceph osd pool create rbd 8 2024-06-14T22:37:27.685 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:27 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/1575456357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-14T22:37:27.685 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:27 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:27.491+0000 7f726b294ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-14T22:37:27.685 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:27 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:27.560+0000 7f726b294ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-14T22:37:27.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:27 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/1575456357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-14T22:37:27.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:27 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/1575456357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-14T22:37:27.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:27 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:27.509+0000 7f8d10873ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-14T22:37:27.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:27 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:27.582+0000 7f8d10873ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-14T22:37:27.787 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:37:28.016 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:27 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:27.718+0000 7f8d10873ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-14T22:37:28.024 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:27 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:27.688+0000 7f726b294ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-14T22:37:28.391 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:28 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:28.104+0000 7f726b294ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-14T22:37:28.442 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:28 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:28.186+0000 7f8d10873ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-14T22:37:28.774 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:28 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:28.395+0000 7f726b294ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-14T22:37:28.775 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:28 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:28.461+0000 7f726b294ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-14T22:37:28.775 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:28 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:28.589+0000 7f726b294ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-14T22:37:28.942 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:28 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:28.498+0000 7f8d10873ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-14T22:37:28.943 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:28 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:28.568+0000 7f8d10873ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-14T22:37:28.943 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:28 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:28.705+0000 7f8d10873ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-14T22:37:29.145 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:28 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:28.845+0000 7f726b294ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-14T22:37:29.145 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:28 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:28.904+0000 7f726b294ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-14T22:37:29.145 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:29 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:29.086+0000 7f726b294ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-14T22:37:29.242 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:28 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:28.977+0000 7f8d10873ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-14T22:37:29.243 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:29 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:29.044+0000 7f8d10873ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-14T22:37:29.499 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:29 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:29.245+0000 7f8d10873ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-14T22:37:29.499 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:29 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:29.315+0000 7f8d10873ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-14T22:37:29.499 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:29 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:29.404+0000 7f8d10873ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-14T22:37:29.499 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:29 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:29.467+0000 7f8d10873ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-14T22:37:29.524 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:29 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:29.148+0000 7f726b294ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-14T22:37:29.525 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:29 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:29.230+0000 7f726b294ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-14T22:37:29.525 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:29 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:29.288+0000 7f726b294ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-14T22:37:30.157 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'rbd' created 2024-06-14T22:37:30.159 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:29 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:29.813+0000 7f726b294ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-14T22:37:30.159 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:29 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:29.951+0000 7f726b294ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-14T22:37:30.159 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:29 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:29.951+0000 7f726b294ec0 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: k8sevents 2024-06-14T22:37:30.159 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:29 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: [14/Jun/2024:22:37:29] ENGINE Bus STARTING 2024-06-14T22:37:30.159 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:29 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: CherryPy Checker: 2024-06-14T22:37:30.159 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:29 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: The Application mounted at '' has an empty config. 2024-06-14T22:37:30.159 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:29 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: 2024-06-14T22:37:30.159 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:30 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: [14/Jun/2024:22:37:30] ENGINE Serving on http://:::9283 2024-06-14T22:37:30.159 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:30 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: [14/Jun/2024:22:37:30] ENGINE Bus STARTED 2024-06-14T22:37:30.362 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/247504300' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]: dispatch 2024-06-14T22:37:30.362 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-mon[38286]: from='mgr.? 172.21.15.178:0/1176793237' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-14T22:37:30.362 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-mon[38286]: from='mgr.? 172.21.15.178:0/1176793237' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-14T22:37:30.362 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-mon[38286]: from='mgr.? 172.21.15.178:0/1176793237' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-14T22:37:30.362 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-mon[38286]: from='mgr.? 172.21.15.178:0/1176793237' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-14T22:37:30.362 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-mon[38286]: Standby manager daemon x restarted 2024-06-14T22:37:30.363 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-mon[38286]: Standby manager daemon x started 2024-06-14T22:37:30.363 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/247504300' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]: dispatch 2024-06-14T22:37:30.363 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-mon[29586]: from='mgr.? 172.21.15.178:0/1176793237' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-14T22:37:30.363 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-mon[29586]: from='mgr.? 172.21.15.178:0/1176793237' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-14T22:37:30.363 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-mon[29586]: from='mgr.? 172.21.15.178:0/1176793237' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-14T22:37:30.363 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-mon[29586]: from='mgr.? 172.21.15.178:0/1176793237' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-14T22:37:30.363 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-mon[29586]: Standby manager daemon x restarted 2024-06-14T22:37:30.363 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-mon[29586]: Standby manager daemon x started 2024-06-14T22:37:30.364 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:30.029+0000 7f8d10873ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-14T22:37:30.364 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:30.182+0000 7f8d10873ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-14T22:37:30.364 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:30.182+0000 7f8d10873ec0 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: k8sevents 2024-06-14T22:37:30.364 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:37:30] ENGINE Bus STARTING 2024-06-14T22:37:30.364 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: CherryPy Checker: 2024-06-14T22:37:30.364 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: The Application mounted at '' has an empty config. 2024-06-14T22:37:30.364 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:30.431 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:30 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/247504300' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]: dispatch 2024-06-14T22:37:30.431 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:30 smithi178 ceph-mon[32942]: from='mgr.? 172.21.15.178:0/1176793237' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-06-14T22:37:30.431 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:30 smithi178 ceph-mon[32942]: from='mgr.? 172.21.15.178:0/1176793237' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-14T22:37:30.431 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:30 smithi178 ceph-mon[32942]: from='mgr.? 172.21.15.178:0/1176793237' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-06-14T22:37:30.432 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:30 smithi178 ceph-mon[32942]: from='mgr.? 172.21.15.178:0/1176793237' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-14T22:37:30.432 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:30 smithi178 ceph-mon[32942]: Standby manager daemon x restarted 2024-06-14T22:37:30.432 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:30 smithi178 ceph-mon[32942]: Standby manager daemon x started 2024-06-14T22:37:30.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:37:30] ENGINE Serving on http://:::9283 2024-06-14T22:37:30.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:30 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:37:30] ENGINE Bus STARTED 2024-06-14T22:37:30.837 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-06-14T22:37:31.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: Failed to load ceph-mgr modules: k8sevents 2024-06-14T22:37:31.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/247504300' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]': finished 2024-06-14T22:37:31.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: osdmap e60: 8 total, 8 up, 8 in 2024-06-14T22:37:31.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: mgrmap e17: y(active, since 4m), standbys: x 2024-06-14T22:37:31.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: Active manager daemon y restarted 2024-06-14T22:37:31.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: Activating manager daemon y 2024-06-14T22:37:31.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: osdmap e61: 8 total, 8 up, 8 in 2024-06-14T22:37:31.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: mgrmap e18: y(active, starting, since 0.00843747s), standbys: x 2024-06-14T22:37:31.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:37:31.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:37:31.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:37:31.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-14T22:37:31.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-14T22:37:31.161 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-14T22:37:31.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-14T22:37:31.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-14T22:37:31.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-14T22:37:31.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:37:31.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-14T22:37:31.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:37:31.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-14T22:37:31.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-14T22:37:31.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-14T22:37:31.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-14T22:37:31.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: Manager daemon y is now available 2024-06-14T22:37:31.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:37:31.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:37:31.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:31.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:37:31.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-14T22:37:31.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-14T22:37:31.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: Failed to load ceph-mgr modules: k8sevents 2024-06-14T22:37:31.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/247504300' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]': finished 2024-06-14T22:37:31.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: osdmap e60: 8 total, 8 up, 8 in 2024-06-14T22:37:31.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: mgrmap e17: y(active, since 4m), standbys: x 2024-06-14T22:37:31.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: Active manager daemon y restarted 2024-06-14T22:37:31.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: Activating manager daemon y 2024-06-14T22:37:31.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: osdmap e61: 8 total, 8 up, 8 in 2024-06-14T22:37:31.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: mgrmap e18: y(active, starting, since 0.00843747s), standbys: x 2024-06-14T22:37:31.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:37:31.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:37:31.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:37:31.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-14T22:37:31.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-14T22:37:31.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-14T22:37:31.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-14T22:37:31.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-14T22:37:31.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-14T22:37:31.446 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:37:31.446 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-14T22:37:31.446 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:37:31.446 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-14T22:37:31.446 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-14T22:37:31.446 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-14T22:37:31.446 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-14T22:37:31.446 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: Manager daemon y is now available 2024-06-14T22:37:31.446 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:37:31.446 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:37:31.446 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:31.447 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:37:31.447 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-14T22:37:31.447 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-14T22:37:31.447 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: Failed to load ceph-mgr modules: k8sevents 2024-06-14T22:37:31.447 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/247504300' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]': finished 2024-06-14T22:37:31.447 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: osdmap e60: 8 total, 8 up, 8 in 2024-06-14T22:37:31.447 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: mgrmap e17: y(active, since 4m), standbys: x 2024-06-14T22:37:31.447 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: Active manager daemon y restarted 2024-06-14T22:37:31.447 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: Activating manager daemon y 2024-06-14T22:37:31.448 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: osdmap e61: 8 total, 8 up, 8 in 2024-06-14T22:37:31.448 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: mgrmap e18: y(active, starting, since 0.00843747s), standbys: x 2024-06-14T22:37:31.448 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-14T22:37:31.448 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-14T22:37:31.448 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-14T22:37:31.448 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-14T22:37:31.448 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-14T22:37:31.448 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-14T22:37:31.448 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-14T22:37:31.448 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-14T22:37:31.448 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-14T22:37:31.448 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-14T22:37:31.448 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-14T22:37:31.449 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-14T22:37:31.449 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-14T22:37:31.449 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-14T22:37:31.449 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-14T22:37:31.449 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-14T22:37:31.449 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: Manager daemon y is now available 2024-06-14T22:37:31.450 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:37:31.450 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:37:31.450 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:31.450 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:37:31.450 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-06-14T22:37:31.450 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-06-14T22:37:31.856 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:37:32.278 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-mon[29586]: mgrmap e19: y(active, since 1.01064s), standbys: x 2024-06-14T22:37:32.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-mon[29586]: osdmap e62: 8 total, 8 up, 8 in 2024-06-14T22:37:32.279 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-mon[29586]: pgmap v4: 9 pgs: 2 creating+activating, 7 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:32.279 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-mon[38286]: mgrmap e19: y(active, since 1.01064s), standbys: x 2024-06-14T22:37:32.279 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-mon[38286]: osdmap e62: 8 total, 8 up, 8 in 2024-06-14T22:37:32.279 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-mon[38286]: pgmap v4: 9 pgs: 2 creating+activating, 7 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:32.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:32 smithi178 ceph-mon[32942]: mgrmap e19: y(active, since 1.01064s), standbys: x 2024-06-14T22:37:32.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:32 smithi178 ceph-mon[32942]: osdmap e62: 8 total, 8 up, 8 in 2024-06-14T22:37:32.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:32 smithi178 ceph-mon[32942]: pgmap v4: 9 pgs: 2 creating+activating, 7 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:33.190 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:32.974+0000 7f8caf6cb640 -1 log_channel(cephadm) log [ERR] : [14/Jun/2024:22:37:32] ENGINE Error in HTTPServer.serve 2024-06-14T22:37:33.190 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: Traceback (most recent call last): 2024-06-14T22:37:33.190 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-14T22:37:33.190 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: self._connections.run(self.expiration_interval) 2024-06-14T22:37:33.190 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-14T22:37:33.190 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: self._run(expiration_interval) 2024-06-14T22:37:33.190 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-14T22:37:33.190 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: new_conn = self._from_server_socket(self.server.socket) 2024-06-14T22:37:33.190 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-14T22:37:33.191 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-14T22:37:33.191 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-14T22:37:33.191 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: s = self.context.wrap_socket( 2024-06-14T22:37:33.191 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-14T22:37:33.191 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: return self.sslsocket_class._create( 2024-06-14T22:37:33.191 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-14T22:37:33.191 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: self.do_handshake() 2024-06-14T22:37:33.191 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-14T22:37:33.191 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: self._sslobj.do_handshake() 2024-06-14T22:37:33.191 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-14T22:37:33.191 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:32 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:37:33.442 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:33 smithi046 ceph-mon[29586]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-14T22:37:33.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:33 smithi046 ceph-mon[29586]: pgmap v5: 9 pgs: 2 creating+activating, 7 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:33.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:33 smithi046 ceph-mon[29586]: osdmap e63: 8 total, 8 up, 8 in 2024-06-14T22:37:33.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:33 smithi046 ceph-mon[29586]: mgrmap e20: y(active, since 2s), standbys: x 2024-06-14T22:37:33.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:33 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:33.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:33 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:33.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:33 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:33.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:33 smithi046 ceph-mon[38286]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-14T22:37:33.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:33 smithi046 ceph-mon[38286]: pgmap v5: 9 pgs: 2 creating+activating, 7 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:33.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:33 smithi046 ceph-mon[38286]: osdmap e63: 8 total, 8 up, 8 in 2024-06-14T22:37:33.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:33 smithi046 ceph-mon[38286]: mgrmap e20: y(active, since 2s), standbys: x 2024-06-14T22:37:33.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:33 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:33.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:33 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:33.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:33 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:33.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:33 smithi178 ceph-mon[32942]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-14T22:37:33.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:33 smithi178 ceph-mon[32942]: pgmap v5: 9 pgs: 2 creating+activating, 7 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:33.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:33 smithi178 ceph-mon[32942]: osdmap e63: 8 total, 8 up, 8 in 2024-06-14T22:37:33.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:33 smithi178 ceph-mon[32942]: mgrmap e20: y(active, since 2s), standbys: x 2024-06-14T22:37:33.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:33 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:33.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:33 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:33.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:33 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:34.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:34 smithi178 ceph-mon[32942]: [14/Jun/2024:22:37:32] ENGINE Bus STARTING 2024-06-14T22:37:34.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:34 smithi178 ceph-mon[32942]: [14/Jun/2024:22:37:32] ENGINE Serving on https://172.21.15.46:7150 2024-06-14T22:37:34.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:34 smithi178 ceph-mon[32942]: [14/Jun/2024:22:37:32] ENGINE Bus STARTED 2024-06-14T22:37:34.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:34 smithi178 ceph-mon[32942]: [14/Jun/2024:22:37:32] ENGINE Error in HTTPServer.serve 2024-06-14T22:37:34.525 INFO:journalctl@ceph.mon.b.smithi178.stdout: Traceback (most recent call last): 2024-06-14T22:37:34.525 INFO:journalctl@ceph.mon.b.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-14T22:37:34.526 INFO:journalctl@ceph.mon.b.smithi178.stdout: self._connections.run(self.expiration_interval) 2024-06-14T22:37:34.526 INFO:journalctl@ceph.mon.b.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-14T22:37:34.526 INFO:journalctl@ceph.mon.b.smithi178.stdout: self._run(expiration_interval) 2024-06-14T22:37:34.526 INFO:journalctl@ceph.mon.b.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-14T22:37:34.526 INFO:journalctl@ceph.mon.b.smithi178.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-06-14T22:37:34.526 INFO:journalctl@ceph.mon.b.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-14T22:37:34.526 INFO:journalctl@ceph.mon.b.smithi178.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-14T22:37:34.526 INFO:journalctl@ceph.mon.b.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-14T22:37:34.526 INFO:journalctl@ceph.mon.b.smithi178.stdout: s = self.context.wrap_socket( 2024-06-14T22:37:34.526 INFO:journalctl@ceph.mon.b.smithi178.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-14T22:37:34.526 INFO:journalctl@ceph.mon.b.smithi178.stdout: return self.sslsocket_class._create( 2024-06-14T22:37:34.526 INFO:journalctl@ceph.mon.b.smithi178.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-14T22:37:34.527 INFO:journalctl@ceph.mon.b.smithi178.stdout: self.do_handshake() 2024-06-14T22:37:34.527 INFO:journalctl@ceph.mon.b.smithi178.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-14T22:37:34.527 INFO:journalctl@ceph.mon.b.smithi178.stdout: self._sslobj.do_handshake() 2024-06-14T22:37:34.527 INFO:journalctl@ceph.mon.b.smithi178.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-14T22:37:34.527 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:34 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:34.527 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:34 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:34.527 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:34 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:34.527 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:34 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:34.527 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:34 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:34.527 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:34 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:34.527 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:34 smithi178 ceph-mon[32942]: Adjusting osd_memory_target on smithi178 to 4016M 2024-06-14T22:37:34.527 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:34 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:34.528 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:34 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:34.528 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:34 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:34.528 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:34 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/1248475249' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]: dispatch 2024-06-14T22:37:34.528 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:34 smithi178 ceph-mon[32942]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]: dispatch 2024-06-14T22:37:34.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[29586]: [14/Jun/2024:22:37:32] ENGINE Bus STARTING 2024-06-14T22:37:34.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[29586]: [14/Jun/2024:22:37:32] ENGINE Serving on https://172.21.15.46:7150 2024-06-14T22:37:34.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[29586]: [14/Jun/2024:22:37:32] ENGINE Bus STARTED 2024-06-14T22:37:34.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[29586]: [14/Jun/2024:22:37:32] ENGINE Error in HTTPServer.serve 2024-06-14T22:37:34.693 INFO:journalctl@ceph.mon.a.smithi046.stdout: Traceback (most recent call last): 2024-06-14T22:37:34.693 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-14T22:37:34.693 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._connections.run(self.expiration_interval) 2024-06-14T22:37:34.693 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-14T22:37:34.694 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._run(expiration_interval) 2024-06-14T22:37:34.694 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-14T22:37:34.694 INFO:journalctl@ceph.mon.a.smithi046.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-06-14T22:37:34.694 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-14T22:37:34.694 INFO:journalctl@ceph.mon.a.smithi046.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-14T22:37:34.694 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-14T22:37:34.694 INFO:journalctl@ceph.mon.a.smithi046.stdout: s = self.context.wrap_socket( 2024-06-14T22:37:34.694 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-14T22:37:34.694 INFO:journalctl@ceph.mon.a.smithi046.stdout: return self.sslsocket_class._create( 2024-06-14T22:37:34.694 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-14T22:37:34.694 INFO:journalctl@ceph.mon.a.smithi046.stdout: self.do_handshake() 2024-06-14T22:37:34.694 INFO:journalctl@ceph.mon.a.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-14T22:37:34.695 INFO:journalctl@ceph.mon.a.smithi046.stdout: self._sslobj.do_handshake() 2024-06-14T22:37:34.695 INFO:journalctl@ceph.mon.a.smithi046.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-14T22:37:34.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:34.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:34.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:34.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:34.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:34.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:34.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[29586]: Adjusting osd_memory_target on smithi178 to 4016M 2024-06-14T22:37:34.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:34.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:34.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:34.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/1248475249' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]: dispatch 2024-06-14T22:37:34.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[29586]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]: dispatch 2024-06-14T22:37:34.696 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[38286]: [14/Jun/2024:22:37:32] ENGINE Bus STARTING 2024-06-14T22:37:34.696 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[38286]: [14/Jun/2024:22:37:32] ENGINE Serving on https://172.21.15.46:7150 2024-06-14T22:37:34.696 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[38286]: [14/Jun/2024:22:37:32] ENGINE Bus STARTED 2024-06-14T22:37:34.696 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[38286]: [14/Jun/2024:22:37:32] ENGINE Error in HTTPServer.serve 2024-06-14T22:37:34.696 INFO:journalctl@ceph.mon.c.smithi046.stdout: Traceback (most recent call last): 2024-06-14T22:37:34.696 INFO:journalctl@ceph.mon.c.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-06-14T22:37:34.696 INFO:journalctl@ceph.mon.c.smithi046.stdout: self._connections.run(self.expiration_interval) 2024-06-14T22:37:34.696 INFO:journalctl@ceph.mon.c.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-06-14T22:37:34.696 INFO:journalctl@ceph.mon.c.smithi046.stdout: self._run(expiration_interval) 2024-06-14T22:37:34.697 INFO:journalctl@ceph.mon.c.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-06-14T22:37:34.697 INFO:journalctl@ceph.mon.c.smithi046.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-06-14T22:37:34.697 INFO:journalctl@ceph.mon.c.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-06-14T22:37:34.697 INFO:journalctl@ceph.mon.c.smithi046.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-06-14T22:37:34.697 INFO:journalctl@ceph.mon.c.smithi046.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-06-14T22:37:34.697 INFO:journalctl@ceph.mon.c.smithi046.stdout: s = self.context.wrap_socket( 2024-06-14T22:37:34.697 INFO:journalctl@ceph.mon.c.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-06-14T22:37:34.697 INFO:journalctl@ceph.mon.c.smithi046.stdout: return self.sslsocket_class._create( 2024-06-14T22:37:34.697 INFO:journalctl@ceph.mon.c.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-06-14T22:37:34.697 INFO:journalctl@ceph.mon.c.smithi046.stdout: self.do_handshake() 2024-06-14T22:37:34.697 INFO:journalctl@ceph.mon.c.smithi046.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-06-14T22:37:34.697 INFO:journalctl@ceph.mon.c.smithi046.stdout: self._sslobj.do_handshake() 2024-06-14T22:37:34.698 INFO:journalctl@ceph.mon.c.smithi046.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1129) 2024-06-14T22:37:34.698 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:34.698 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:34.698 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:34.698 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:34.698 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:34.698 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-14T22:37:34.698 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[38286]: Adjusting osd_memory_target on smithi178 to 4016M 2024-06-14T22:37:34.698 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:34.698 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:34.698 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:34.698 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/1248475249' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]: dispatch 2024-06-14T22:37:34.699 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:34 smithi046 ceph-mon[38286]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]: dispatch 2024-06-14T22:37:34.886 INFO:teuthology.orchestra.run.smithi046.stderr:enabled application 'rbd' on pool 'rbd' 2024-06-14T22:37:35.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:35 smithi178 ceph-mon[32942]: pgmap v7: 9 pgs: 2 creating+activating, 7 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:35.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:35 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:35.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:35 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:35.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:35 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:37:35.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:35 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:37:35.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:35 smithi178 ceph-mon[32942]: mgrmap e21: y(active, since 4s), standbys: x 2024-06-14T22:37:35.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:35 smithi178 ceph-mon[32942]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]': finished 2024-06-14T22:37:35.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:35 smithi178 ceph-mon[32942]: osdmap e64: 8 total, 8 up, 8 in 2024-06-14T22:37:35.662 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:35 smithi046 ceph-mon[38286]: pgmap v7: 9 pgs: 2 creating+activating, 7 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:35.662 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:35 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:35.662 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:35 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:35.662 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:35 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:37:35.662 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:35 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:37:35.663 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:35 smithi046 ceph-mon[38286]: mgrmap e21: y(active, since 4s), standbys: x 2024-06-14T22:37:35.663 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:35 smithi046 ceph-mon[38286]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]': finished 2024-06-14T22:37:35.663 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:35 smithi046 ceph-mon[38286]: osdmap e64: 8 total, 8 up, 8 in 2024-06-14T22:37:35.663 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:35 smithi046 ceph-mon[29586]: pgmap v7: 9 pgs: 2 creating+activating, 7 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:35.663 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:35 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:35.663 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:35 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:35.663 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:35 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:37:35.663 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:35 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:37:35.663 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:35 smithi046 ceph-mon[29586]: mgrmap e21: y(active, since 4s), standbys: x 2024-06-14T22:37:35.663 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:35 smithi046 ceph-mon[29586]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]': finished 2024-06-14T22:37:35.664 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:35 smithi046 ceph-mon[29586]: osdmap e64: 8 total, 8 up, 8 in 2024-06-14T22:37:35.689 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-14T22:37:35.689 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-14T22:37:35.689 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph mgr dump --format=json 2024-06-14T22:37:35.960 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:37:36.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[29586]: Updating smithi046:/etc/ceph/ceph.conf 2024-06-14T22:37:36.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[29586]: Updating smithi178:/etc/ceph/ceph.conf 2024-06-14T22:37:36.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[29586]: Updating smithi046:/var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-14T22:37:36.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[29586]: Updating smithi178:/var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-14T22:37:36.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[29586]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-06-14T22:37:36.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[29586]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-06-14T22:37:36.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[29586]: Updating smithi178:/var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-14T22:37:36.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[29586]: Updating smithi046:/var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-14T22:37:36.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:36.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:36.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:36.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:36.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:36.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[29586]: Deploying daemon alertmanager.a on smithi046 2024-06-14T22:37:36.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[38286]: Updating smithi046:/etc/ceph/ceph.conf 2024-06-14T22:37:36.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[38286]: Updating smithi178:/etc/ceph/ceph.conf 2024-06-14T22:37:36.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[38286]: Updating smithi046:/var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-14T22:37:36.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[38286]: Updating smithi178:/var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-14T22:37:36.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[38286]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-06-14T22:37:36.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[38286]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-06-14T22:37:36.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[38286]: Updating smithi178:/var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-14T22:37:36.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[38286]: Updating smithi046:/var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-14T22:37:36.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:36.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:36.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:36.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:36.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:36.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:36 smithi046 ceph-mon[38286]: Deploying daemon alertmanager.a on smithi046 2024-06-14T22:37:36.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:36 smithi178 ceph-mon[32942]: Updating smithi046:/etc/ceph/ceph.conf 2024-06-14T22:37:36.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:36 smithi178 ceph-mon[32942]: Updating smithi178:/etc/ceph/ceph.conf 2024-06-14T22:37:36.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:36 smithi178 ceph-mon[32942]: Updating smithi046:/var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-14T22:37:36.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:36 smithi178 ceph-mon[32942]: Updating smithi178:/var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/config/ceph.conf 2024-06-14T22:37:36.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:36 smithi178 ceph-mon[32942]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-06-14T22:37:36.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:36 smithi178 ceph-mon[32942]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-06-14T22:37:36.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:36 smithi178 ceph-mon[32942]: Updating smithi178:/var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-14T22:37:36.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:36 smithi178 ceph-mon[32942]: Updating smithi046:/var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/config/ceph.client.admin.keyring 2024-06-14T22:37:36.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:36 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:36.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:36 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:36.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:36 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:36.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:36 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:36.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:36 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:36.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:36 smithi178 ceph-mon[32942]: Deploying daemon alertmanager.a on smithi046 2024-06-14T22:37:37.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:37 smithi046 ceph-mon[38286]: pgmap v9: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:37.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:37 smithi046 ceph-mon[38286]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-14T22:37:37.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:37 smithi046 ceph-mon[38286]: Cluster is now healthy 2024-06-14T22:37:37.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:37 smithi046 ceph-mon[29586]: pgmap v9: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:37.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:37 smithi046 ceph-mon[29586]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-14T22:37:37.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:37 smithi046 ceph-mon[29586]: Cluster is now healthy 2024-06-14T22:37:37.775 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:37 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:37:37] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-06-14T22:37:37.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:37 smithi178 ceph-mon[32942]: pgmap v9: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:37.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:37 smithi178 ceph-mon[32942]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-14T22:37:37.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:37 smithi178 ceph-mon[32942]: Cluster is now healthy 2024-06-14T22:37:38.454 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:37:39.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:39 smithi046 ceph-mon[38286]: pgmap v10: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-06-14T22:37:39.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:39 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/2379328607' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-14T22:37:39.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:39 smithi046 ceph-mon[29586]: pgmap v10: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-06-14T22:37:39.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:39 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/2379328607' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-14T22:37:39.732 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"flags":0,"active_gid":14415,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":835295015},{"type":"v1","addr":"172.21.15.46:6801","nonce":835295015}]},"active_addr":"172.21.15.46:6801/835295015","active_change":"2024-06-14T22:37:30.185431+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24355,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":false,"error_string":"No module named 'kubernetes.client.models.v1_event'","module_options":{}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":false,"error_string":"No module named 'kubernetes.client.models.v1_event'","module_options":{}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.46:8443/","prometheus":"http://172.21.15.46:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":61,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":664260966}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":556015621}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":3331427503}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":245604168}]}]} 2024-06-14T22:37:39.736 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-14T22:37:39.736 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-14T22:37:39.736 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd dump --format=json 2024-06-14T22:37:39.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:39 smithi178 ceph-mon[32942]: pgmap v10: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-06-14T22:37:39.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:39 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/2379328607' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-14T22:37:40.276 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:37:41.604 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:41 smithi046 ceph-mon[29586]: pgmap v11: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 14 op/s 2024-06-14T22:37:41.604 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:41 smithi046 ceph-mon[38286]: pgmap v11: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 14 op/s 2024-06-14T22:37:41.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:41 smithi178 ceph-mon[32942]: pgmap v11: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 14 op/s 2024-06-14T22:37:43.585 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:37:43.585 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":64,"fsid":"cb009720-2a9d-11ef-bc9d-c7b262605968","created":"2024-06-14T22:31:50.709367+0000","modified":"2024-06-14T22:37:34.881054+0000","last_up_change":"2024-06-14T22:36:57.461962+0000","last_in_change":"2024-06-14T22:36:40.661606+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-14T22:34:58.841776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-06-14T22:37:30.106154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"33945598-f864-40e6-904e-1bff72ec8b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6803","nonce":2467893990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6805","nonce":2467893990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6809","nonce":2467893990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6807","nonce":2467893990}]},"public_addr":"172.21.15.46:6803/2467893990","cluster_addr":"172.21.15.46:6805/2467893990","heartbeat_back_addr":"172.21.15.46:6809/2467893990","heartbeat_front_addr":"172.21.15.46:6807/2467893990","state":["exists","up"]},{"osd":1,"uuid":"48f0e5ef-2d0a-4858-a629-69f82c604f9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6811","nonce":2133775041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6813","nonce":2133775041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6817","nonce":2133775041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6815","nonce":2133775041}]},"public_addr":"172.21.15.46:6811/2133775041","cluster_addr":"172.21.15.46:6813/2133775041","heartbeat_back_addr":"172.21.15.46:6817/2133775041","heartbeat_front_addr":"172.21.15.46:6815/2133775041","state":["exists","up"]},{"osd":2,"uuid":"74b998f5-fb99-4c37-9463-11dd602f1364","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6819","nonce":3036855066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6821","nonce":3036855066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6825","nonce":3036855066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6823","nonce":3036855066}]},"public_addr":"172.21.15.46:6819/3036855066","cluster_addr":"172.21.15.46:6821/3036855066","heartbeat_back_addr":"172.21.15.46:6825/3036855066","heartbeat_front_addr":"172.21.15.46:6823/3036855066","state":["exists","up"]},{"osd":3,"uuid":"e4bd323c-2885-47a0-a44c-275077fc4b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6827","nonce":2597052465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6829","nonce":2597052465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6833","nonce":2597052465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6831","nonce":2597052465}]},"public_addr":"172.21.15.46:6827/2597052465","cluster_addr":"172.21.15.46:6829/2597052465","heartbeat_back_addr":"172.21.15.46:6833/2597052465","heartbeat_front_addr":"172.21.15.46:6831/2597052465","state":["exists","up"]},{"osd":4,"uuid":"0a0b2d3d-3e61-47be-a95f-8294f79ecc7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6801","nonce":2179289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6803","nonce":2179289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6807","nonce":2179289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6805","nonce":2179289524}]},"public_addr":"172.21.15.178:6801/2179289524","cluster_addr":"172.21.15.178:6803/2179289524","heartbeat_back_addr":"172.21.15.178:6807/2179289524","heartbeat_front_addr":"172.21.15.178:6805/2179289524","state":["exists","up"]},{"osd":5,"uuid":"29f79df4-039f-44a3-8570-7252eca3abe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6809","nonce":2813027226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6811","nonce":2813027226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6815","nonce":2813027226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6813","nonce":2813027226}]},"public_addr":"172.21.15.178:6809/2813027226","cluster_addr":"172.21.15.178:6811/2813027226","heartbeat_back_addr":"172.21.15.178:6815/2813027226","heartbeat_front_addr":"172.21.15.178:6813/2813027226","state":["exists","up"]},{"osd":6,"uuid":"56661a2b-4e13-47e7-972a-bb1861d5e33d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6817","nonce":3396977870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6819","nonce":3396977870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6823","nonce":3396977870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6821","nonce":3396977870}]},"public_addr":"172.21.15.178:6817/3396977870","cluster_addr":"172.21.15.178:6819/3396977870","heartbeat_back_addr":"172.21.15.178:6823/3396977870","heartbeat_front_addr":"172.21.15.178:6821/3396977870","state":["exists","up"]},{"osd":7,"uuid":"205c8b05-359d-4ff3-a158-3e80e1c5fda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6825","nonce":826597267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6827","nonce":826597267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6831","nonce":826597267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6829","nonce":826597267}]},"public_addr":"172.21.15.178:6825/826597267","cluster_addr":"172.21.15.178:6827/826597267","heartbeat_back_addr":"172.21.15.178:6831/826597267","heartbeat_front_addr":"172.21.15.178:6829/826597267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:34:06.575453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:34:31.020820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:34:55.617857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:35:20.862506+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:35:42.941353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:36:06.653807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:36:31.539489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:36:56.167994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/1358765809":"2024-06-15T22:37:30.185199+0000","172.21.15.46:0/1856025406":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/2833589127":"2024-06-15T22:32:44.342121+0000","172.21.15.46:6800/1612763800":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/662686570":"2024-06-15T22:37:30.185199+0000","172.21.15.46:6801/1612763800":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/4205563548":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/1630559323":"2024-06-15T22:32:44.342121+0000","172.21.15.46:6801/873354145":"2024-06-15T22:37:30.185199+0000","172.21.15.46:0/3550067878":"2024-06-15T22:32:44.342121+0000","172.21.15.46:0/2045722581":"2024-06-15T22:37:30.185199+0000","172.21.15.46:0/96950257":"2024-06-15T22:32:22.225646+0000","172.21.15.46:6800/4238739813":"2024-06-15T22:32:44.342121+0000","172.21.15.46:6801/4238739813":"2024-06-15T22:32:44.342121+0000","172.21.15.46:0/2079110403":"2024-06-15T22:37:30.185199+0000","172.21.15.46:6800/873354145":"2024-06-15T22:37:30.185199+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-14T22:37:43.586 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:43 smithi046 ceph-mon[29586]: pgmap v12: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-14T22:37:43.586 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:43 smithi046 ceph-mon[38286]: pgmap v12: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-14T22:37:43.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:43 smithi178 ceph-mon[32942]: pgmap v12: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-14T22:37:44.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:44 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/141586473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-14T22:37:44.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:44 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/141586473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-14T22:37:44.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:44 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/141586473' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-14T22:37:44.825 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-14T22:37:44.825 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd dump --format=json 2024-06-14T22:37:45.541 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:37:45.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:45 smithi046 ceph-mon[29586]: pgmap v13: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-14T22:37:45.631 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:45 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:37:45.631 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:45 smithi046 ceph-mon[38286]: pgmap v13: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-14T22:37:45.631 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:45 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:37:45.774 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:45 smithi178 ceph-mon[32942]: pgmap v13: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 11 op/s 2024-06-14T22:37:45.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:45 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:37:47.221 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:37:47 smithi046 podman[69428]: 2024-06-14 22:37:47.225346725 +0000 UTC m=+0.215284345 container create d72e10fbe29210e219f855edafbff3e1c1590090ea7cb915d560c13f03857dab (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-14T22:37:47.573 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:47 smithi046 ceph-mon[29586]: pgmap v14: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-14T22:37:47.574 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:47 smithi046 ceph-mon[38286]: pgmap v14: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-14T22:37:47.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:47 smithi178 ceph-mon[32942]: pgmap v14: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-06-14T22:37:47.775 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:47 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:37:47] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-06-14T22:37:47.853 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:37:47 smithi046 podman[69428]: 2024-06-14 22:37:47.738511795 +0000 UTC m=+0.728449428 container init d72e10fbe29210e219f855edafbff3e1c1590090ea7cb915d560c13f03857dab (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-14T22:37:47.853 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:37:47 smithi046 podman[69428]: 2024-06-14 22:37:47.744898828 +0000 UTC m=+0.734836449 container start d72e10fbe29210e219f855edafbff3e1c1590090ea7cb915d560c13f03857dab (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-14T22:37:47.853 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:37:47 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[69455]: ts=2024-06-14T22:37:47.759Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-06-14T22:37:47.853 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:37:47 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[69455]: ts=2024-06-14T22:37:47.759Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-06-14T22:37:47.853 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:37:47 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[69455]: ts=2024-06-14T22:37:47.760Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.46 port=9094 2024-06-14T22:37:47.853 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:37:47 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[69455]: ts=2024-06-14T22:37:47.761Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-06-14T22:37:47.853 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:37:47 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[69455]: ts=2024-06-14T22:37:47.785Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-14T22:37:47.853 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:37:47 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[69455]: ts=2024-06-14T22:37:47.786Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-14T22:37:47.854 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:37:47 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[69455]: ts=2024-06-14T22:37:47.789Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-06-14T22:37:47.854 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:37:47 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[69455]: ts=2024-06-14T22:37:47.789Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-06-14T22:37:48.140 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:37:48 smithi046 bash[69428]: d72e10fbe29210e219f855edafbff3e1c1590090ea7cb915d560c13f03857dab 2024-06-14T22:37:48.140 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:37:48 smithi046 systemd[1]: Started Ceph alertmanager.a for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:37:48.140 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:37:48.141 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":64,"fsid":"cb009720-2a9d-11ef-bc9d-c7b262605968","created":"2024-06-14T22:31:50.709367+0000","modified":"2024-06-14T22:37:34.881054+0000","last_up_change":"2024-06-14T22:36:57.461962+0000","last_in_change":"2024-06-14T22:36:40.661606+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-14T22:34:58.841776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-06-14T22:37:30.106154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"33945598-f864-40e6-904e-1bff72ec8b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6803","nonce":2467893990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6805","nonce":2467893990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6809","nonce":2467893990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6807","nonce":2467893990}]},"public_addr":"172.21.15.46:6803/2467893990","cluster_addr":"172.21.15.46:6805/2467893990","heartbeat_back_addr":"172.21.15.46:6809/2467893990","heartbeat_front_addr":"172.21.15.46:6807/2467893990","state":["exists","up"]},{"osd":1,"uuid":"48f0e5ef-2d0a-4858-a629-69f82c604f9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6811","nonce":2133775041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6813","nonce":2133775041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6817","nonce":2133775041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6815","nonce":2133775041}]},"public_addr":"172.21.15.46:6811/2133775041","cluster_addr":"172.21.15.46:6813/2133775041","heartbeat_back_addr":"172.21.15.46:6817/2133775041","heartbeat_front_addr":"172.21.15.46:6815/2133775041","state":["exists","up"]},{"osd":2,"uuid":"74b998f5-fb99-4c37-9463-11dd602f1364","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6819","nonce":3036855066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6821","nonce":3036855066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6825","nonce":3036855066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6823","nonce":3036855066}]},"public_addr":"172.21.15.46:6819/3036855066","cluster_addr":"172.21.15.46:6821/3036855066","heartbeat_back_addr":"172.21.15.46:6825/3036855066","heartbeat_front_addr":"172.21.15.46:6823/3036855066","state":["exists","up"]},{"osd":3,"uuid":"e4bd323c-2885-47a0-a44c-275077fc4b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6827","nonce":2597052465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6829","nonce":2597052465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6833","nonce":2597052465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6831","nonce":2597052465}]},"public_addr":"172.21.15.46:6827/2597052465","cluster_addr":"172.21.15.46:6829/2597052465","heartbeat_back_addr":"172.21.15.46:6833/2597052465","heartbeat_front_addr":"172.21.15.46:6831/2597052465","state":["exists","up"]},{"osd":4,"uuid":"0a0b2d3d-3e61-47be-a95f-8294f79ecc7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6801","nonce":2179289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6803","nonce":2179289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6807","nonce":2179289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6805","nonce":2179289524}]},"public_addr":"172.21.15.178:6801/2179289524","cluster_addr":"172.21.15.178:6803/2179289524","heartbeat_back_addr":"172.21.15.178:6807/2179289524","heartbeat_front_addr":"172.21.15.178:6805/2179289524","state":["exists","up"]},{"osd":5,"uuid":"29f79df4-039f-44a3-8570-7252eca3abe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6809","nonce":2813027226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6811","nonce":2813027226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6815","nonce":2813027226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6813","nonce":2813027226}]},"public_addr":"172.21.15.178:6809/2813027226","cluster_addr":"172.21.15.178:6811/2813027226","heartbeat_back_addr":"172.21.15.178:6815/2813027226","heartbeat_front_addr":"172.21.15.178:6813/2813027226","state":["exists","up"]},{"osd":6,"uuid":"56661a2b-4e13-47e7-972a-bb1861d5e33d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6817","nonce":3396977870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6819","nonce":3396977870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6823","nonce":3396977870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6821","nonce":3396977870}]},"public_addr":"172.21.15.178:6817/3396977870","cluster_addr":"172.21.15.178:6819/3396977870","heartbeat_back_addr":"172.21.15.178:6823/3396977870","heartbeat_front_addr":"172.21.15.178:6821/3396977870","state":["exists","up"]},{"osd":7,"uuid":"205c8b05-359d-4ff3-a158-3e80e1c5fda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6825","nonce":826597267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6827","nonce":826597267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6831","nonce":826597267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6829","nonce":826597267}]},"public_addr":"172.21.15.178:6825/826597267","cluster_addr":"172.21.15.178:6827/826597267","heartbeat_back_addr":"172.21.15.178:6831/826597267","heartbeat_front_addr":"172.21.15.178:6829/826597267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:34:06.575453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:34:31.020820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:34:55.617857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:35:20.862506+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:35:42.941353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:36:06.653807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:36:31.539489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:36:56.167994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/1358765809":"2024-06-15T22:37:30.185199+0000","172.21.15.46:0/1856025406":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/2833589127":"2024-06-15T22:32:44.342121+0000","172.21.15.46:6800/1612763800":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/662686570":"2024-06-15T22:37:30.185199+0000","172.21.15.46:6801/1612763800":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/4205563548":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/1630559323":"2024-06-15T22:32:44.342121+0000","172.21.15.46:6801/873354145":"2024-06-15T22:37:30.185199+0000","172.21.15.46:0/3550067878":"2024-06-15T22:32:44.342121+0000","172.21.15.46:0/2045722581":"2024-06-15T22:37:30.185199+0000","172.21.15.46:0/96950257":"2024-06-15T22:32:22.225646+0000","172.21.15.46:6800/4238739813":"2024-06-15T22:32:44.342121+0000","172.21.15.46:6801/4238739813":"2024-06-15T22:32:44.342121+0000","172.21.15.46:0/2079110403":"2024-06-15T22:37:30.185199+0000","172.21.15.46:6800/873354145":"2024-06-15T22:37:30.185199+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-14T22:37:48.391 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:48 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:37:48] ENGINE Bus STOPPING 2024-06-14T22:37:48.391 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:48 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:37:48] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-14T22:37:48.391 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:48 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:37:48] ENGINE Bus STOPPED 2024-06-14T22:37:48.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:48 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:37:48] ENGINE Bus STARTING 2024-06-14T22:37:48.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:48 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:37:48] ENGINE Serving on http://:::9283 2024-06-14T22:37:48.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:37:48 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:37:48] ENGINE Bus STARTED 2024-06-14T22:37:48.816 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-14T22:37:48.816 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-14T22:37:48.816 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-14T22:37:48.817 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-14T22:37:48.817 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-14T22:37:48.818 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-14T22:37:48.818 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-14T22:37:48.818 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-14T22:37:49.369 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:37:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:37:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:37:49.370 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:37:49.372 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:37:49.373 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:37:49.373 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:37:49.380 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:37:49.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/869596597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-14T22:37:49.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-14T22:37:49.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[29586]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-14T22:37:49.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[29586]: pgmap v15: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-14T22:37:49.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[29586]: Deploying daemon grafana.a on smithi178 2024-06-14T22:37:49.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/869596597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-14T22:37:49.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-14T22:37:49.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[38286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-14T22:37:49.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[38286]: pgmap v15: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-14T22:37:49.445 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-mon[38286]: Deploying daemon grafana.a on smithi178 2024-06-14T22:37:49.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:49 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:49 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:49 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:49 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:49 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/869596597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-14T22:37:49.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:49 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:49 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:49 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-14T22:37:49.526 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:49 smithi178 ceph-mon[32942]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-06-14T22:37:49.526 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:49 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:49.526 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:49 smithi178 ceph-mon[32942]: pgmap v15: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-06-14T22:37:49.526 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:49 smithi178 ceph-mon[32942]: Deploying daemon grafana.a on smithi178 2024-06-14T22:37:49.779 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:37:49 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[69455]: ts=2024-06-14T22:37:49.762Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000078715s 2024-06-14T22:37:51.518 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:51 smithi046 ceph-mon[38286]: pgmap v16: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:51.518 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:51 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:51.518 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:51 smithi046 ceph-mon[29586]: pgmap v16: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:51.518 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:51 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:51.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:51 smithi178 ceph-mon[32942]: pgmap v16: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:51.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:51 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:37:53.490 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:53 smithi046 ceph-mon[29586]: pgmap v17: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:53.490 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:53 smithi046 ceph-mon[38286]: pgmap v17: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:53.524 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:53 smithi178 ceph-mon[32942]: pgmap v17: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:55.426 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:55 smithi046 ceph-mon[29586]: pgmap v18: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:55.426 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:55 smithi046 ceph-mon[38286]: pgmap v18: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:55.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:55 smithi178 ceph-mon[32942]: pgmap v18: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:57.460 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:57 smithi046 ceph-mon[29586]: pgmap v19: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:57.460 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:57 smithi046 ceph-mon[38286]: pgmap v19: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:57.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:57 smithi178 ceph-mon[32942]: pgmap v19: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:58.025 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:37:57 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:37:57] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-06-14T22:37:58.060 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:37:57 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[69455]: ts=2024-06-14T22:37:57.764Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002493405s 2024-06-14T22:37:59.113 INFO:teuthology.orchestra.run.smithi046.stdout:201863462931 2024-06-14T22:37:59.114 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-14T22:37:59.388 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:37:59 smithi178 ceph-mon[32942]: pgmap v20: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:59.436 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:37:59 smithi046 ceph-mon[29586]: pgmap v20: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:37:59.436 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:37:59 smithi046 ceph-mon[38286]: pgmap v20: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:00.249 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:00 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:38:00.326 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:00 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:38:00.326 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:00 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:38:01.071 INFO:teuthology.orchestra.run.smithi046.stdout:167503724568 2024-06-14T22:38:01.072 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-14T22:38:01.358 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:01 smithi178 ceph-mon[32942]: pgmap v21: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:01.415 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:01 smithi046 ceph-mon[29586]: pgmap v21: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:01.416 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:01 smithi046 ceph-mon[38286]: pgmap v21: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:01.810 INFO:teuthology.orchestra.run.smithi046.stdout:77309411366 2024-06-14T22:38:01.811 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-14T22:38:02.474 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.477679201Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-06-14T22:38:02.474 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.478294837Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-06-14T22:38:02.775 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.478669846Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-06-14T22:38:02.775 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.479063152Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-06-14T22:38:02.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.479450673Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-06-14T22:38:02.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.480955959Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-06-14T22:38:02.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.481329175Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-06-14T22:38:02.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.481727745Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-06-14T22:38:02.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.481998555Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-06-14T22:38:02.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.482325831Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-06-14T22:38:02.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.482573786Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-06-14T22:38:02.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.483208216Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-06-14T22:38:02.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.483830922Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-06-14T22:38:02.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.484239344Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-06-14T22:38:02.777 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.484629771Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-06-14T22:38:02.777 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.484884364Z level=info msg="Executing migration" id="Add epoch_end column" 2024-06-14T22:38:02.777 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.485534898Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-06-14T22:38:02.777 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.485933223Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-06-14T22:38:02.777 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.486261276Z level=info msg="Executing migration" id="Move region to single row" 2024-06-14T22:38:02.777 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.486551901Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-06-14T22:38:02.777 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.486884015Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-06-14T22:38:02.777 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.487254384Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-06-14T22:38:02.777 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.487649166Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-06-14T22:38:02.777 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.488023109Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-06-14T22:38:02.777 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.488398036Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-06-14T22:38:02.777 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.48874946Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-06-14T22:38:02.778 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.489008191Z level=info msg="Executing migration" id="create test_data table" 2024-06-14T22:38:02.778 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.489382154Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-06-14T22:38:02.778 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.489734889Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-06-14T22:38:02.778 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.490140432Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-06-14T22:38:02.778 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.490535206Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-06-14T22:38:02.778 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.490794703Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-06-14T22:38:02.778 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.491109608Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-06-14T22:38:02.778 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.491369683Z level=info msg="Executing migration" id="create team table" 2024-06-14T22:38:02.778 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.491710222Z level=info msg="Executing migration" id="add index team.org_id" 2024-06-14T22:38:02.778 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.49219408Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-06-14T22:38:02.778 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.492590806Z level=info msg="Executing migration" id="create team member table" 2024-06-14T22:38:02.779 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.492945145Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-06-14T22:38:02.779 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.493363539Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-06-14T22:38:02.781 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.493753311Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-06-14T22:38:02.781 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.494153044Z level=info msg="Executing migration" id="Add column email to team table" 2024-06-14T22:38:02.781 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.49480836Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-06-14T22:38:02.781 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.495474651Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-06-14T22:38:02.781 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.496144504Z level=info msg="Executing migration" id="create dashboard acl table" 2024-06-14T22:38:02.781 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.496510838Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-06-14T22:38:02.781 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.496925268Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-06-14T22:38:02.781 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.49739799Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-06-14T22:38:02.781 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.497828828Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-06-14T22:38:02.781 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.498517685Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-06-14T22:38:02.782 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.498947015Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-06-14T22:38:02.782 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.499427284Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-06-14T22:38:02.782 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.499862111Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-06-14T22:38:02.782 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.500275617Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-06-14T22:38:02.782 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.500591015Z level=info msg="Executing migration" id="create tag table" 2024-06-14T22:38:02.782 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.500989815Z level=info msg="Executing migration" id="add index tag.key_value" 2024-06-14T22:38:02.782 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.501396223Z level=info msg="Executing migration" id="create login attempt table" 2024-06-14T22:38:02.782 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.501802002Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-06-14T22:38:02.782 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.502235175Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-06-14T22:38:02.782 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.502662391Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-06-14T22:38:02.782 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.504524271Z level=info msg="Executing migration" id="create login_attempt v2" 2024-06-14T22:38:02.783 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.504886867Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-06-14T22:38:02.783 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.505311803Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-06-14T22:38:02.783 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.505599746Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-06-14T22:38:02.783 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.505900851Z level=info msg="Executing migration" id="create user auth table" 2024-06-14T22:38:02.783 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.506260603Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-06-14T22:38:02.783 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.50668123Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-06-14T22:38:02.783 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.506912978Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-06-14T22:38:02.783 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.507685442Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-06-14T22:38:02.783 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.50844032Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-06-14T22:38:02.783 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.509171782Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-06-14T22:38:02.783 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.509875936Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-06-14T22:38:02.783 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.510279104Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-06-14T22:38:02.784 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.510999124Z level=info msg="Executing migration" id="create server_lock table" 2024-06-14T22:38:02.784 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.511367017Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-06-14T22:38:02.784 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.511799231Z level=info msg="Executing migration" id="create user auth token table" 2024-06-14T22:38:02.784 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.512232965Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-06-14T22:38:02.784 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.512638858Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-06-14T22:38:02.784 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.513023869Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-06-14T22:38:02.784 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.513401237Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-06-14T22:38:02.784 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.514110581Z level=info msg="Executing migration" id="create cache_data table" 2024-06-14T22:38:02.784 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.514470917Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-06-14T22:38:02.784 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.514847712Z level=info msg="Executing migration" id="create short_url table v1" 2024-06-14T22:38:02.784 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.515241836Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-06-14T22:38:02.784 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.51564187Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-06-14T22:38:02.785 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.515864212Z level=info msg="Executing migration" id="delete alert_definition table" 2024-06-14T22:38:02.785 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.516139189Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-06-14T22:38:02.785 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.516556125Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-06-14T22:38:02.785 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.516932887Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-06-14T22:38:02.785 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.51734939Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-06-14T22:38:02.785 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.51757711Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-06-14T22:38:02.785 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.517909075Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-06-14T22:38:02.785 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.518278925Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-06-14T22:38:02.785 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.51865919Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-06-14T22:38:02.785 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.519040497Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-06-14T22:38:02.785 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.519791151Z level=info msg="Executing migration" id="drop alert_definition table" 2024-06-14T22:38:02.785 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.520204339Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-06-14T22:38:02.786 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.52047067Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-06-14T22:38:02.786 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.520805979Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-06-14T22:38:02.786 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.521201761Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-06-14T22:38:02.786 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.521577356Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-06-14T22:38:02.786 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.521799306Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-06-14T22:38:02.786 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.522203189Z level=info msg="Executing migration" id="create alert_instance table" 2024-06-14T22:38:02.786 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.522573458Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-06-14T22:38:02.786 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.522921875Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-06-14T22:38:02.786 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.523313675Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-06-14T22:38:02.786 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.5240197Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-06-14T22:38:02.786 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.52442778Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-06-14T22:38:02.787 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.524758834Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-06-14T22:38:02.787 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.528076542Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-06-14T22:38:02.787 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.531365923Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-06-14T22:38:02.787 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.531757119Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-06-14T22:38:02.787 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.532143661Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-06-14T22:38:02.787 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.532896496Z level=info msg="Executing migration" id="create alert_rule table" 2024-06-14T22:38:02.787 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.533281016Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-06-14T22:38:02.787 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.533663955Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-06-14T22:38:02.787 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.534052368Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-06-14T22:38:02.787 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.534433538Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-06-14T22:38:02.787 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.534659009Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-06-14T22:38:02.787 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.535453752Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-06-14T22:38:02.788 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.536243114Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-06-14T22:38:02.788 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.537026576Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-06-14T22:38:02.788 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.537402547Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-06-14T22:38:02.788 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.537753791Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-06-14T22:38:02.788 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.538569434Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-06-14T22:38:02.788 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.539422166Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-06-14T22:38:02.788 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.53980654Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-06-14T22:38:02.788 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.54060157Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-06-14T22:38:02.788 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.541455843Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-06-14T22:38:02.788 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.541726742Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-06-14T22:38:02.788 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.542134364Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-06-14T22:38:02.788 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.542581477Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-06-14T22:38:02.789 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.543027468Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-06-14T22:38:02.789 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.54330878Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-06-14T22:38:02.789 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.544142907Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-06-14T22:38:02.789 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.544930475Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-06-14T22:38:02.789 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.545722982Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-06-14T22:38:02.789 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.546548954Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-06-14T22:38:02.789 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.547424391Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-06-14T22:38:02.789 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.547697742Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-06-14T22:38:02.789 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.548059554Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-06-14T22:38:02.789 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.54884862Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-06-14T22:38:02.789 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.549142035Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-06-14T22:38:02.789 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.549997863Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-06-14T22:38:02.790 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.550401966Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-06-14T22:38:02.790 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.551219553Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-06-14T22:38:02.790 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.551584784Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-06-14T22:38:02.790 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.551992669Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-06-14T22:38:02.790 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.552818425Z level=info msg="Executing migration" id="create provenance_type table" 2024-06-14T22:38:02.790 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.553195982Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-06-14T22:38:02.790 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.55359962Z level=info msg="Executing migration" id="create alert_image table" 2024-06-14T22:38:02.790 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.553956634Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-06-14T22:38:02.790 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.554359178Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-06-14T22:38:02.790 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.554602558Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-06-14T22:38:02.790 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.554981317Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-06-14T22:38:02.791 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.555364628Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-06-14T22:38:02.791 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.555447755Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-06-14T22:38:02.791 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.555664306Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-06-14T22:38:02.791 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.555960752Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-06-14T22:38:02.791 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.556417796Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-06-14T22:38:02.791 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.55654056Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-06-14T22:38:02.791 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.556661949Z level=info msg="alerts found to migrate" alerts=0 2024-06-14T22:38:02.791 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.557009642Z level=info msg="Executing migration" id="create library_element table v1" 2024-06-14T22:38:02.791 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.557446645Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-06-14T22:38:02.791 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.557852869Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-06-14T22:38:02.791 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.558227521Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-06-14T22:38:02.791 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.558630819Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-06-14T22:38:02.792 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.559096911Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-06-14T22:38:02.792 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.559361745Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-06-14T22:38:02.792 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.559666283Z level=info msg="Executing migration" id="create data_keys table" 2024-06-14T22:38:02.792 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.560052613Z level=info msg="Executing migration" id="create secrets table" 2024-06-14T22:38:02.792 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.560467608Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-06-14T22:38:02.792 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.564629911Z level=info msg="Executing migration" id="add name column into data_keys" 2024-06-14T22:38:02.792 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.565498268Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-06-14T22:38:02.792 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.565779695Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-06-14T22:38:02.792 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.569872051Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-06-14T22:38:02.792 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.574040268Z level=info msg="Executing migration" id="create kv_store table v1" 2024-06-14T22:38:02.792 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.574418306Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-06-14T22:38:02.792 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.574824687Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-06-14T22:38:02.793 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.57513657Z level=info msg="Executing migration" id="create permission table" 2024-06-14T22:38:02.793 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.575486572Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-06-14T22:38:02.793 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.575864911Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-06-14T22:38:02.793 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.576260123Z level=info msg="Executing migration" id="create role table" 2024-06-14T22:38:02.793 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.57661162Z level=info msg="Executing migration" id="add column display_name" 2024-06-14T22:38:02.793 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.577520514Z level=info msg="Executing migration" id="add column group_name" 2024-06-14T22:38:02.793 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.578428408Z level=info msg="Executing migration" id="add index role.org_id" 2024-06-14T22:38:02.793 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.57881926Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-06-14T22:38:02.793 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.579231373Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-06-14T22:38:02.793 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.579620297Z level=info msg="Executing migration" id="create team role table" 2024-06-14T22:38:02.793 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.579961552Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-06-14T22:38:02.793 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.580364147Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-06-14T22:38:02.794 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.580769519Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-06-14T22:38:02.794 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.581163039Z level=info msg="Executing migration" id="create user role table" 2024-06-14T22:38:02.794 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.581511184Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-06-14T22:38:02.794 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.581888653Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-06-14T22:38:02.794 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.582277936Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-06-14T22:38:02.794 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.582662251Z level=info msg="Executing migration" id="create builtin role table" 2024-06-14T22:38:02.794 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.583053388Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-06-14T22:38:02.794 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.583486574Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-06-14T22:38:02.794 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.583874611Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-06-14T22:38:02.794 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.584806213Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-06-14T22:38:02.794 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.585244622Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-06-14T22:38:02.794 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.585645264Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-06-14T22:38:02.795 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.585993598Z level=info msg="Executing migration" id="add unique index role.uid" 2024-06-14T22:38:02.795 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.586382066Z level=info msg="Executing migration" id="create seed assignment table" 2024-06-14T22:38:02.795 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.586719286Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-06-14T22:38:02.795 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.587120957Z level=info msg="Executing migration" id="add column hidden to role table" 2024-06-14T22:38:02.795 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.588024048Z level=info msg="Executing migration" id="create query_history table v1" 2024-06-14T22:38:02.795 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.588390133Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-06-14T22:38:02.795 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.588804958Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-06-14T22:38:02.795 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.589064085Z level=info msg="Executing migration" id="teams permissions migration" 2024-06-14T22:38:02.795 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.589418009Z level=info msg="Executing migration" id="dashboard permissions" 2024-06-14T22:38:02.795 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.589788364Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-06-14T22:38:02.795 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.590117797Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-06-14T22:38:02.795 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.590381233Z level=info msg="Executing migration" id="alerting notification permissions" 2024-06-14T22:38:02.796 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.590659078Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-06-14T22:38:02.796 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.591086041Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-06-14T22:38:02.796 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.591586427Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-06-14T22:38:02.796 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.592524321Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-06-14T22:38:02.796 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.592788512Z level=info msg="Executing migration" id="create correlation table v1" 2024-06-14T22:38:02.796 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.593172592Z level=info msg="Executing migration" id="add index correlations.uid" 2024-06-14T22:38:02.796 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.593564714Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-06-14T22:38:02.796 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.593978397Z level=info msg="Executing migration" id="add correlation config column" 2024-06-14T22:38:02.796 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.594906175Z level=info msg="Executing migration" id="create entity_events table" 2024-06-14T22:38:02.796 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.595319726Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-06-14T22:38:02.796 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.595691297Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-06-14T22:38:02.796 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.595767822Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-06-14T22:38:02.796 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.596019577Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-06-14T22:38:02.797 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.596097122Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-06-14T22:38:02.797 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.596329979Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-06-14T22:38:02.797 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.596675113Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-06-14T22:38:02.797 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.597014908Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-06-14T22:38:02.797 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.597448563Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-06-14T22:38:02.797 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.597863966Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-06-14T22:38:02.797 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.598264878Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-06-14T22:38:02.797 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.598667382Z level=info msg="Executing migration" id="Drop public config table" 2024-06-14T22:38:02.797 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.599043021Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-06-14T22:38:02.797 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.599452254Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-06-14T22:38:02.797 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.599815099Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-06-14T22:38:02.797 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.600248728Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-06-14T22:38:02.797 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.600660944Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-06-14T22:38:02.797 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.603612859Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-06-14T22:38:02.798 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.604580193Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-06-14T22:38:02.798 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.605580283Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-06-14T22:38:02.798 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.605856352Z level=info msg="Executing migration" id="add share column" 2024-06-14T22:38:02.798 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.606806306Z level=info msg="Executing migration" id="create default alerting folders" 2024-06-14T22:38:02.798 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.607233241Z level=info msg="Executing migration" id="create file table" 2024-06-14T22:38:02.798 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.607669874Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-06-14T22:38:02.798 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.608070054Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-06-14T22:38:02.798 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.608493022Z level=info msg="Executing migration" id="create file_meta table" 2024-06-14T22:38:02.798 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.608833312Z level=info msg="Executing migration" id="file table idx: path key" 2024-06-14T22:38:02.798 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.609230472Z level=info msg="Executing migration" id="set path collation in file table" 2024-06-14T22:38:02.798 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.609479277Z level=info msg="Executing migration" id="managed permissions migration" 2024-06-14T22:38:02.798 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.609775849Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-06-14T22:38:02.799 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.610052993Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-06-14T22:38:02.799 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.61045451Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-06-14T22:38:02.799 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.611463127Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-06-14T22:38:02.799 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.611757619Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-06-14T22:38:02.799 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.612162666Z level=info msg="Executing migration" id="update group index for alert rules" 2024-06-14T22:38:02.799 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.612527679Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-06-14T22:38:02.799 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.612772694Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-06-14T22:38:02.799 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.613100782Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-06-14T22:38:02.799 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.61407915Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-06-14T22:38:02.799 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.615066318Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-06-14T22:38:02.799 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.615473252Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-06-14T22:38:02.799 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.625212236Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-06-14T22:38:02.800 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.625637Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-06-14T22:38:02.800 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.62621461Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-06-14T22:38:02.800 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.629690982Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-06-14T22:38:02.800 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.629980835Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-06-14T22:38:02.800 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.63028135Z level=info msg="Executing migration" id="create folder table" 2024-06-14T22:38:02.800 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.630602895Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-06-14T22:38:02.800 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.630999991Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-06-14T22:38:02.800 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.63139511Z level=info msg="Executing migration" id="Update folder title length" 2024-06-14T22:38:02.800 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.63161386Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-06-14T22:38:02.800 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=migrator t=2024-06-14T22:38:02.632013664Z level=info msg="migrations completed" performed=485 skipped=0 duration=238.254497ms 2024-06-14T22:38:02.800 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=sqlstore t=2024-06-14T22:38:02.632411145Z level=info msg="Created default organization" 2024-06-14T22:38:02.800 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=plugin.loader t=2024-06-14T22:38:02.686418086Z level=info msg="Plugin registered" pluginID=input 2024-06-14T22:38:02.800 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=plugin.loader t=2024-06-14T22:38:02.712901423Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-06-14T22:38:02.800 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=plugin.loader t=2024-06-14T22:38:02.712919987Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-06-14T22:38:02.801 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=secrets t=2024-06-14T22:38:02.712950614Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-06-14T22:38:02.801 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=query_data t=2024-06-14T22:38:02.716357172Z level=info msg="Query Service initialization" 2024-06-14T22:38:02.801 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:02 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=live.push_http t=2024-06-14T22:38:02.719029524Z level=info msg="Live Push Gateway initialization" 2024-06-14T22:38:02.883 INFO:teuthology.orchestra.run.smithi046.stdout:55834574891 2024-06-14T22:38:02.884 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-14T22:38:03.599 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:03 smithi046 ceph-mon[38286]: pgmap v22: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:03.599 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:03 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:03.599 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:03 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:03.599 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:03 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:03.599 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:03 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:03.599 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:03 smithi046 ceph-mon[38286]: Deploying daemon node-exporter.a on smithi046 2024-06-14T22:38:03.599 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:03 smithi046 ceph-mon[29586]: pgmap v22: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:03.599 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:03 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:03.600 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:03 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:03.600 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:03 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:03.600 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:03 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:03.600 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:03 smithi046 ceph-mon[29586]: Deploying daemon node-exporter.a on smithi046 2024-06-14T22:38:03.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-mon[32942]: pgmap v22: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:03.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:03.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:03.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:03.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:03.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-mon[32942]: Deploying daemon node-exporter.a on smithi046 2024-06-14T22:38:03.775 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=infra.usagestats.collector t=2024-06-14T22:38:03.428052901Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-06-14T22:38:03.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=provisioning.datasources t=2024-06-14T22:38:03.428351852Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-06-14T22:38:03.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=provisioning.datasources t=2024-06-14T22:38:03.461060567Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-06-14T22:38:03.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=provisioning.datasources t=2024-06-14T22:38:03.492505702Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-06-14T22:38:03.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=provisioning.alerting t=2024-06-14T22:38:03.514137098Z level=info msg="starting to provision alerting" 2024-06-14T22:38:03.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=provisioning.alerting t=2024-06-14T22:38:03.514157637Z level=info msg="finished to provision alerting" 2024-06-14T22:38:03.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=grafanaStorageLogger t=2024-06-14T22:38:03.514327606Z level=info msg="storage starting" 2024-06-14T22:38:03.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=ngalert.state.manager t=2024-06-14T22:38:03.514363434Z level=info msg="Warming state cache for startup" 2024-06-14T22:38:03.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=http.server t=2024-06-14T22:38:03.516156038Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-06-14T22:38:03.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=sqlstore.transactions t=2024-06-14T22:38:03.525145924Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-06-14T22:38:03.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=ngalert.state.manager t=2024-06-14T22:38:03.525156118Z level=info msg="State cache has been initialized" states=0 duration=10.792172ms 2024-06-14T22:38:03.776 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=ticker t=2024-06-14T22:38:03.525217044Z level=info msg=starting first_tick=2024-06-14T22:38:10Z 2024-06-14T22:38:03.777 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=ngalert.multiorg.alertmanager t=2024-06-14T22:38:03.525222275Z level=info msg="starting MultiOrg Alertmanager" 2024-06-14T22:38:03.777 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=sqlstore.transactions t=2024-06-14T22:38:03.575809006Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-06-14T22:38:03.777 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=sqlstore.transactions t=2024-06-14T22:38:03.586229006Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-06-14T22:38:03.777 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:38:03 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a[63684]: logger=sqlstore.transactions t=2024-06-14T22:38:03.622164436Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-06-14T22:38:04.696 INFO:teuthology.orchestra.run.smithi046.stdout:133143986205 2024-06-14T22:38:04.696 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-14T22:38:05.654 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:05 smithi046 ceph-mon[29586]: pgmap v23: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:05.655 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:05 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:05.655 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:05 smithi046 ceph-mon[38286]: pgmap v23: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:05.655 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:05 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:05.677 INFO:teuthology.orchestra.run.smithi046.stdout:34359738417 2024-06-14T22:38:05.678 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-14T22:38:05.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:05 smithi178 ceph-mon[32942]: pgmap v23: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:05.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:05 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:06.741 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:38:06.741 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:38:06.757 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:38:06.774 INFO:teuthology.orchestra.run.smithi046.stdout:111669149730 2024-06-14T22:38:06.775 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-14T22:38:07.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:07 smithi046 ceph-mon[29586]: pgmap v24: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:07.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:07 smithi046 ceph-mon[38286]: pgmap v24: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:07.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:07 smithi178 ceph-mon[32942]: pgmap v24: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:07.775 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:38:07 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:38:07] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-06-14T22:38:08.671 INFO:teuthology.orchestra.run.smithi046.stdout:236223201295 2024-06-14T22:38:08.671 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-14T22:38:09.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:09 smithi046 ceph-mon[29586]: pgmap v25: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:09.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:09 smithi046 ceph-mon[38286]: pgmap v25: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:09.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:09 smithi178 ceph-mon[32942]: pgmap v25: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:09.852 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:38:09.852 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:38:09.852 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:38:10.443 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:10 smithi046 bash[71908]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-06-14T22:38:10.943 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:10 smithi046 bash[71908]: Writing manifest to image destination 2024-06-14T22:38:11.295 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:11 smithi046 podman[71908]: 2024-06-14 22:38:10.982073373 +0000 UTC m=+3.185353304 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-06-14T22:38:11.629 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:11 smithi046 ceph-mon[38286]: pgmap v26: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:11.629 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:11 smithi046 ceph-mon[29586]: pgmap v26: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:11.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:11 smithi178 ceph-mon[32942]: pgmap v26: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:11.887 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:11 smithi046 podman[71908]: 2024-06-14 22:38:11.744956834 +0000 UTC m=+3.948236761 container create 7ccc5883403b21008de7788b9b600a6bd0447fdc692e42ea8a5c885f55787daa (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-14T22:38:12.170 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:11 smithi046 podman[71908]: 2024-06-14 22:38:11.98942352 +0000 UTC m=+4.192703461 container init 7ccc5883403b21008de7788b9b600a6bd0447fdc692e42ea8a5c885f55787daa (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-14T22:38:12.170 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:11 smithi046 podman[71908]: 2024-06-14 22:38:11.995890026 +0000 UTC m=+4.199169954 container start 7ccc5883403b21008de7788b9b600a6bd0447fdc692e42ea8a5c885f55787daa (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-14T22:38:12.170 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.000Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-06-14T22:38:12.170 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.000Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-06-14T22:38:12.170 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-06-14T22:38:12.170 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-06-14T22:38:12.170 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-06-14T22:38:12.170 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-06-14T22:38:12.170 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-06-14T22:38:12.171 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=arp 2024-06-14T22:38:12.171 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=bcache 2024-06-14T22:38:12.171 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=bonding 2024-06-14T22:38:12.171 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=btrfs 2024-06-14T22:38:12.171 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=conntrack 2024-06-14T22:38:12.171 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=cpu 2024-06-14T22:38:12.171 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-06-14T22:38:12.172 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=diskstats 2024-06-14T22:38:12.172 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=dmi 2024-06-14T22:38:12.172 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=edac 2024-06-14T22:38:12.172 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=entropy 2024-06-14T22:38:12.172 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-06-14T22:38:12.172 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=filefd 2024-06-14T22:38:12.172 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=filesystem 2024-06-14T22:38:12.172 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=hwmon 2024-06-14T22:38:12.172 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=infiniband 2024-06-14T22:38:12.173 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=ipvs 2024-06-14T22:38:12.173 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=loadavg 2024-06-14T22:38:12.173 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=mdadm 2024-06-14T22:38:12.173 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=meminfo 2024-06-14T22:38:12.173 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=netclass 2024-06-14T22:38:12.173 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=netdev 2024-06-14T22:38:12.173 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=netstat 2024-06-14T22:38:12.173 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=nfs 2024-06-14T22:38:12.173 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=nfsd 2024-06-14T22:38:12.173 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=nvme 2024-06-14T22:38:12.173 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=os 2024-06-14T22:38:12.173 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-06-14T22:38:12.174 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=pressure 2024-06-14T22:38:12.174 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=rapl 2024-06-14T22:38:12.174 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=schedstat 2024-06-14T22:38:12.174 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=selinux 2024-06-14T22:38:12.174 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=sockstat 2024-06-14T22:38:12.174 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=softnet 2024-06-14T22:38:12.174 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=stat 2024-06-14T22:38:12.174 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=tapestats 2024-06-14T22:38:12.174 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=textfile 2024-06-14T22:38:12.175 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-06-14T22:38:12.175 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=time 2024-06-14T22:38:12.175 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-06-14T22:38:12.175 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=uname 2024-06-14T22:38:12.175 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=vmstat 2024-06-14T22:38:12.175 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=xfs 2024-06-14T22:38:12.175 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.001Z caller=node_exporter.go:117 level=info collector=zfs 2024-06-14T22:38:12.175 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.002Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-06-14T22:38:12.175 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a[72198]: ts=2024-06-14T22:38:12.002Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-06-14T22:38:12.175 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 bash[71908]: 7ccc5883403b21008de7788b9b600a6bd0447fdc692e42ea8a5c885f55787daa 2024-06-14T22:38:12.175 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:38:12 smithi046 systemd[1]: Started Ceph node-exporter.a for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:38:12.781 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:38:13.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:13 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:13.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:13 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:13.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:13 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:13.283 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:13 smithi046 ceph-mon[29586]: Deploying daemon node-exporter.b on smithi178 2024-06-14T22:38:13.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:13 smithi046 ceph-mon[29586]: pgmap v27: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:13.284 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:13 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:13.284 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:13 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:13.284 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:13 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:13.284 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:13 smithi046 ceph-mon[38286]: Deploying daemon node-exporter.b on smithi178 2024-06-14T22:38:13.284 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:13 smithi046 ceph-mon[38286]: pgmap v27: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:13.499 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:13 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:13.499 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:13 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:13.499 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:13 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:13.499 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:13 smithi178 ceph-mon[32942]: Deploying daemon node-exporter.b on smithi178 2024-06-14T22:38:13.499 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:13 smithi178 ceph-mon[32942]: pgmap v27: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:13.756 INFO:teuthology.orchestra.run.smithi046.stdout:201863462935 2024-06-14T22:38:13.775 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:13 smithi178 bash[63954]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-06-14T22:38:14.414 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:14 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/57434316' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-14T22:38:14.414 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:14 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/2920747063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-14T22:38:14.414 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:14 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/57434316' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-14T22:38:14.414 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:14 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/2920747063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-14T22:38:14.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:14 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/57434316' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-14T22:38:14.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:14 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/2920747063' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-14T22:38:14.525 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:14 smithi178 bash[63954]: Getting image source signatures 2024-06-14T22:38:14.525 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:14 smithi178 bash[63954]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-06-14T22:38:14.525 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:14 smithi178 bash[63954]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-06-14T22:38:14.526 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:14 smithi178 bash[63954]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-06-14T22:38:14.545 INFO:teuthology.orchestra.run.smithi046.stdout:55834574894 2024-06-14T22:38:15.402 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:15 smithi046 ceph-mon[29586]: pgmap v28: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:15.402 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:15 smithi046 ceph-mon[38286]: pgmap v28: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:15.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:15 smithi178 ceph-mon[32942]: pgmap v28: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:16.024 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:15 smithi178 bash[63954]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-06-14T22:38:16.024 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:15 smithi178 bash[63954]: Writing manifest to image destination 2024-06-14T22:38:16.162 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574891 got 55834574894 for osd.1 2024-06-14T22:38:16.163 DEBUG:teuthology.parallel:result is None 2024-06-14T22:38:16.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:38:16.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/4083620368' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-14T22:38:16.275 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 podman[63954]: 2024-06-14 22:38:15.996126425 +0000 UTC m=+2.392744997 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-06-14T22:38:16.275 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 podman[63954]: 2024-06-14 22:38:16.103270695 +0000 UTC m=+2.499889252 container create 03c1664adb5aed8c0e6ad5f666f5175f69fa718639019cdf029a1fc48b5cf1d3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-14T22:38:16.299 INFO:teuthology.orchestra.run.smithi046.stdout:77309411369 2024-06-14T22:38:16.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:16 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:38:16.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:16 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/4083620368' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-14T22:38:16.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:16 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:38:16.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:16 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/4083620368' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-14T22:38:16.775 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 podman[63954]: 2024-06-14 22:38:16.372011037 +0000 UTC m=+2.768629610 container init 03c1664adb5aed8c0e6ad5f666f5175f69fa718639019cdf029a1fc48b5cf1d3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-14T22:38:16.775 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 podman[63954]: 2024-06-14 22:38:16.37739421 +0000 UTC m=+2.774012769 container start 03c1664adb5aed8c0e6ad5f666f5175f69fa718639019cdf029a1fc48b5cf1d3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-14T22:38:16.775 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.381Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-06-14T22:38:16.775 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.381Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-06-14T22:38:16.775 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.381Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-06-14T22:38:16.776 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.381Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-06-14T22:38:16.776 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.381Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-06-14T22:38:16.776 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-06-14T22:38:16.776 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-06-14T22:38:16.776 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=arp 2024-06-14T22:38:16.776 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=bcache 2024-06-14T22:38:16.776 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=bonding 2024-06-14T22:38:16.776 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=btrfs 2024-06-14T22:38:16.776 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=conntrack 2024-06-14T22:38:16.776 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=cpu 2024-06-14T22:38:16.776 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-06-14T22:38:16.777 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=diskstats 2024-06-14T22:38:16.777 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=dmi 2024-06-14T22:38:16.777 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=edac 2024-06-14T22:38:16.777 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=entropy 2024-06-14T22:38:16.777 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-06-14T22:38:16.777 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=filefd 2024-06-14T22:38:16.777 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=filesystem 2024-06-14T22:38:16.777 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=hwmon 2024-06-14T22:38:16.777 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=infiniband 2024-06-14T22:38:16.777 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=ipvs 2024-06-14T22:38:16.778 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=loadavg 2024-06-14T22:38:16.778 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=mdadm 2024-06-14T22:38:16.778 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=meminfo 2024-06-14T22:38:16.778 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=netclass 2024-06-14T22:38:16.778 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=netdev 2024-06-14T22:38:16.778 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=netstat 2024-06-14T22:38:16.778 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=nfs 2024-06-14T22:38:16.778 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=nfsd 2024-06-14T22:38:16.778 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=nvme 2024-06-14T22:38:16.778 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=os 2024-06-14T22:38:16.778 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-06-14T22:38:16.778 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=pressure 2024-06-14T22:38:16.779 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=rapl 2024-06-14T22:38:16.779 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=schedstat 2024-06-14T22:38:16.779 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=selinux 2024-06-14T22:38:16.779 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=sockstat 2024-06-14T22:38:16.779 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=softnet 2024-06-14T22:38:16.779 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=stat 2024-06-14T22:38:16.779 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=tapestats 2024-06-14T22:38:16.779 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=textfile 2024-06-14T22:38:16.779 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-06-14T22:38:16.779 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=time 2024-06-14T22:38:16.779 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-06-14T22:38:16.779 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=uname 2024-06-14T22:38:16.779 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=vmstat 2024-06-14T22:38:16.780 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=xfs 2024-06-14T22:38:16.780 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.382Z caller=node_exporter.go:117 level=info collector=zfs 2024-06-14T22:38:16.780 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.383Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-06-14T22:38:16.780 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b[64002]: ts=2024-06-14T22:38:16.383Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-06-14T22:38:16.780 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 bash[63954]: 03c1664adb5aed8c0e6ad5f666f5175f69fa718639019cdf029a1fc48b5cf1d3 2024-06-14T22:38:16.780 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:38:16 smithi178 systemd[1]: Started Ceph node-exporter.b for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:38:16.890 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411366 got 77309411369 for osd.2 2024-06-14T22:38:16.891 DEBUG:teuthology.parallel:result is None 2024-06-14T22:38:17.226 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462931 got 201863462935 for osd.6 2024-06-14T22:38:17.227 DEBUG:teuthology.parallel:result is None 2024-06-14T22:38:17.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:17 smithi046 ceph-mon[29586]: pgmap v29: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:17.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:17 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:17.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:17 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:17.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:17 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:17.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:17 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:17.410 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:17 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:17.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:17 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:38:17.411 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:17 smithi046 ceph-mon[38286]: pgmap v29: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:17.411 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:17 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:17.411 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:17 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:17.411 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:17 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:17.411 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:17 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:17.411 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:17 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:17.411 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:17 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:38:17.446 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:17 smithi178 ceph-mon[32942]: pgmap v29: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:17.446 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:17 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:17.446 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:17 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:17.446 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:17 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:17.446 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:17 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:17.446 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:17 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:17.446 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:17 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:38:17.774 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:38:17 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:38:17] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-06-14T22:38:17.822 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:38:19.157 INFO:teuthology.orchestra.run.smithi046.stdout:111669149733 2024-06-14T22:38:19.345 INFO:teuthology.orchestra.run.smithi046.stdout:133143986208 2024-06-14T22:38:19.346 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:19 smithi046 ceph-mon[29586]: pgmap v30: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:19.346 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:19 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/1680537995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-14T22:38:19.346 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:19 smithi046 ceph-mon[38286]: pgmap v30: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:19.346 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:19 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/1680537995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-14T22:38:19.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:19 smithi178 ceph-mon[32942]: pgmap v30: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:19.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:19 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/1680537995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-14T22:38:20.071 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986205 got 133143986208 for osd.4 2024-06-14T22:38:20.071 DEBUG:teuthology.parallel:result is None 2024-06-14T22:38:20.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:20 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/230960095' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-14T22:38:20.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:20 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:20.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:20 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:20.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:20 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/230960095' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-14T22:38:20.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:20 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:20.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:20 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:20.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:20 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/230960095' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-14T22:38:20.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:20 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:20.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:20 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:21.158 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149730 got 111669149733 for osd.3 2024-06-14T22:38:21.158 DEBUG:teuthology.parallel:result is None 2024-06-14T22:38:21.424 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:21 smithi046 ceph-mon[29586]: pgmap v31: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:21.425 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:21 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:21.425 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:21 smithi046 ceph-mon[38286]: pgmap v31: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:21.425 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:21 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:21.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:21 smithi178 ceph-mon[32942]: pgmap v31: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:21.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:21 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:22.146 INFO:teuthology.orchestra.run.smithi046.stdout:34359738420 2024-06-14T22:38:22.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:22 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/949131223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-14T22:38:22.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:22 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/949131223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-14T22:38:22.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:22 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/949131223' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-14T22:38:22.733 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738417 got 34359738420 for osd.0 2024-06-14T22:38:22.733 DEBUG:teuthology.parallel:result is None 2024-06-14T22:38:23.093 INFO:teuthology.orchestra.run.smithi046.stdout:236223201299 2024-06-14T22:38:23.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:23 smithi046 ceph-mon[38286]: pgmap v32: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:23.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:23 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/2382525440' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-14T22:38:23.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:23 smithi046 ceph-mon[29586]: pgmap v32: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:23.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:23 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/2382525440' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-14T22:38:23.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:23 smithi178 ceph-mon[32942]: pgmap v32: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:23.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:23 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/2382525440' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-14T22:38:24.656 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201295 got 236223201299 for osd.7 2024-06-14T22:38:24.657 DEBUG:teuthology.parallel:result is None 2024-06-14T22:38:25.332 INFO:teuthology.orchestra.run.smithi046.stdout:167503724573 2024-06-14T22:38:25.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:25 smithi178 ceph-mon[32942]: pgmap v33: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:25.633 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:25 smithi046 ceph-mon[29586]: pgmap v33: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:25.634 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:25 smithi046 ceph-mon[38286]: pgmap v33: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:26.062 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724568 got 167503724573 for osd.5 2024-06-14T22:38:26.062 DEBUG:teuthology.parallel:result is None 2024-06-14T22:38:26.062 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-14T22:38:26.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph pg dump --format=json 2024-06-14T22:38:26.353 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:26 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/3933261465' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-14T22:38:26.353 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:26 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/3933261465' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-14T22:38:26.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:26 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/3933261465' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-14T22:38:26.794 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:38:27.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:27 smithi046 ceph-mon[29586]: pgmap v34: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:27.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:27 smithi046 ceph-mon[38286]: pgmap v34: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:27.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:27 smithi178 ceph-mon[32942]: pgmap v34: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:28.025 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:38:27 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:38:27] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-06-14T22:38:29.141 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:38:29.142 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-06-14T22:38:29.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:29 smithi046 ceph-mon[29586]: pgmap v35: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:29.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:29 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:29.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:29 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:29.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:29 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:38:29.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:29 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:38:29.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:29 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:29.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:29 smithi046 ceph-mon[38286]: pgmap v35: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:29.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:29 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:29.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:29 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:29.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:29 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:38:29.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:29 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:38:29.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:29 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:29.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:29 smithi178 ceph-mon[32942]: pgmap v35: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:29.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:29 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:29.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:29 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:29.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:29 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:38:29.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:29 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:38:29.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:29 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:30.678 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:30 smithi046 ceph-mon[29586]: Reconfiguring alertmanager.a (dependencies changed)... 2024-06-14T22:38:30.678 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:30 smithi046 ceph-mon[29586]: Reconfiguring daemon alertmanager.a on smithi046 2024-06-14T22:38:30.678 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:30 smithi046 ceph-mon[29586]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-14T22:38:30.679 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:30 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:38:30.679 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:30 smithi046 ceph-mon[38286]: Reconfiguring alertmanager.a (dependencies changed)... 2024-06-14T22:38:30.679 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:30 smithi046 ceph-mon[38286]: Reconfiguring daemon alertmanager.a on smithi046 2024-06-14T22:38:30.679 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:30 smithi046 ceph-mon[38286]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-14T22:38:30.679 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:30 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:38:30.708 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-06-14T22:38:28.209504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385548,"kb_used_data":6700,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575668,"statfs":{"total":767960285184,"available":765517484032,"internally_reserved":0,"allocated":6860800,"data_stored":3897876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002022"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-06-14T22:37:32.230249+0000","last_change":"2024-06-14T22:37:30.196467+0000","last_active":"2024-06-14T22:37:32.230249+0000","last_peered":"2024-06-14T22:37:32.230249+0000","last_clean":"2024-06-14T22:37:32.230249+0000","last_became_active":"2024-06-14T22:37:30.196165+0000","last_became_peered":"2024-06-14T22:37:30.196165+0000","last_unstale":"2024-06-14T22:37:32.230249+0000","last_undegraded":"2024-06-14T22:37:32.230249+0000","last_fullsized":"2024-06-14T22:37:32.230249+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:27:43.756983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-06-14T22:37:32.586721+0000","last_change":"2024-06-14T22:37:30.198386+0000","last_active":"2024-06-14T22:37:32.586721+0000","last_peered":"2024-06-14T22:37:32.586721+0000","last_clean":"2024-06-14T22:37:32.586721+0000","last_became_active":"2024-06-14T22:37:30.198073+0000","last_became_peered":"2024-06-14T22:37:30.198073+0000","last_unstale":"2024-06-14T22:37:32.586721+0000","last_undegraded":"2024-06-14T22:37:32.586721+0000","last_fullsized":"2024-06-14T22:37:32.586721+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T00:16:05.703306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-06-14T22:37:32.204596+0000","last_change":"2024-06-14T22:37:30.196859+0000","last_active":"2024-06-14T22:37:32.204596+0000","last_peered":"2024-06-14T22:37:32.204596+0000","last_clean":"2024-06-14T22:37:32.204596+0000","last_became_active":"2024-06-14T22:37:30.196536+0000","last_became_peered":"2024-06-14T22:37:30.196536+0000","last_unstale":"2024-06-14T22:37:32.204596+0000","last_undegraded":"2024-06-14T22:37:32.204596+0000","last_fullsized":"2024-06-14T22:37:32.204596+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:46:59.482280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-06-14T22:37:32.586717+0000","last_change":"2024-06-14T22:37:30.197518+0000","last_active":"2024-06-14T22:37:32.586717+0000","last_peered":"2024-06-14T22:37:32.586717+0000","last_clean":"2024-06-14T22:37:32.586717+0000","last_became_active":"2024-06-14T22:37:30.197224+0000","last_became_peered":"2024-06-14T22:37:30.197224+0000","last_unstale":"2024-06-14T22:37:32.586717+0000","last_undegraded":"2024-06-14T22:37:32.586717+0000","last_fullsized":"2024-06-14T22:37:32.586717+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T03:41:54.860308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-06-14T22:37:32.586955+0000","last_change":"2024-06-14T22:37:31.471372+0000","last_active":"2024-06-14T22:37:32.586955+0000","last_peered":"2024-06-14T22:37:32.586955+0000","last_clean":"2024-06-14T22:37:32.586955+0000","last_became_active":"2024-06-14T22:37:31.471087+0000","last_became_peered":"2024-06-14T22:37:31.471087+0000","last_unstale":"2024-06-14T22:37:32.586955+0000","last_undegraded":"2024-06-14T22:37:32.586955+0000","last_fullsized":"2024-06-14T22:37:32.586955+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:31:03.406918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-06-14T22:37:32.204209+0000","last_change":"2024-06-14T22:37:30.327693+0000","last_active":"2024-06-14T22:37:32.204209+0000","last_peered":"2024-06-14T22:37:32.204209+0000","last_clean":"2024-06-14T22:37:32.204209+0000","last_became_active":"2024-06-14T22:37:30.327313+0000","last_became_peered":"2024-06-14T22:37:30.327313+0000","last_unstale":"2024-06-14T22:37:32.204209+0000","last_undegraded":"2024-06-14T22:37:32.204209+0000","last_fullsized":"2024-06-14T22:37:32.204209+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T00:09:29.434302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-06-14T22:37:32.204499+0000","last_change":"2024-06-14T22:37:30.199225+0000","last_active":"2024-06-14T22:37:32.204499+0000","last_peered":"2024-06-14T22:37:32.204499+0000","last_clean":"2024-06-14T22:37:32.204499+0000","last_became_active":"2024-06-14T22:37:30.199011+0000","last_became_peered":"2024-06-14T22:37:30.199011+0000","last_unstale":"2024-06-14T22:37:32.204499+0000","last_undegraded":"2024-06-14T22:37:32.204499+0000","last_fullsized":"2024-06-14T22:37:32.204499+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:45:43.859896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-06-14T22:37:32.586955+0000","last_change":"2024-06-14T22:37:31.278821+0000","last_active":"2024-06-14T22:37:32.586955+0000","last_peered":"2024-06-14T22:37:32.586955+0000","last_clean":"2024-06-14T22:37:32.586955+0000","last_became_active":"2024-06-14T22:37:31.278504+0000","last_became_peered":"2024-06-14T22:37:31.278504+0000","last_unstale":"2024-06-14T22:37:32.586955+0000","last_undegraded":"2024-06-14T22:37:32.586955+0000","last_fullsized":"2024-06-14T22:37:32.586955+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T10:00:23.519181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"63'128","reported_seq":271,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-06-14T22:37:32.255867+0000","last_change":"2024-06-14T22:37:01.481768+0000","last_active":"2024-06-14T22:37:32.255867+0000","last_peered":"2024-06-14T22:37:32.255867+0000","last_clean":"2024-06-14T22:37:32.255867+0000","last_became_active":"2024-06-14T22:37:01.481339+0000","last_became_peered":"2024-06-14T22:37:01.481339+0000","last_unstale":"2024-06-14T22:37:32.255867+0000","last_undegraded":"2024-06-14T22:37:32.255867+0000","last_fullsized":"2024-06-14T22:37:32.255867+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:34:59.252736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:34:59.252736+0000","last_clean_scrub_stamp":"2024-06-14T22:34:59.252736+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T22:55:57.126904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201300,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298552,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446600,"statfs":{"total":95995035648,"available":95689318400,"internally_reserved":0,"allocated":1224704,"data_stored":855906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.41899999999999998}]},{"osd":1,"last update":"Fri Jun 14 22:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.39700000000000002}]},{"osd":2,"last update":"Fri Jun 14 22:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.48199999999999998}]},{"osd":3,"last update":"Fri Jun 14 22:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.37},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.443}]},{"osd":4,"last update":"Fri Jun 14 22:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.42799999999999999}]},{"osd":5,"last update":"Fri Jun 14 22:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.317},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.38}]},{"osd":6,"last update":"Fri Jun 14 22:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.33000000000000002}]}]},{"osd":6,"up_from":47,"seq":201863462938,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298548,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446604,"statfs":{"total":95995035648,"available":95689322496,"internally_reserved":0,"allocated":1220608,"data_stored":855641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:37:35 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.45700000000000002}]},{"osd":1,"last update":"Fri Jun 14 22:37:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Jun 14 22:37:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.495}]},{"osd":3,"last update":"Fri Jun 14 22:37:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.439}]},{"osd":4,"last update":"Fri Jun 14 22:37:35 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.373},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.35999999999999999}]},{"osd":5,"last update":"Fri Jun 14 22:37:35 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.32200000000000001}]},{"osd":7,"last update":"Fri Jun 14 22:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.40100000000000002}]}]},{"osd":5,"up_from":39,"seq":167503724574,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297980,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447172,"statfs":{"total":95995035648,"available":95689904128,"internally_reserved":0,"allocated":638976,"data_stored":266126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.379,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78900000000000003,"5min":0.873,"15min":0.873},"last":0.501}]},{"osd":1,"last update":"Fri Jun 14 22:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.40699999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri Jun 14 22:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51500000000000001,"15min":0.50600000000000001},"min":{"1min":0.39100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.434,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.56999999999999995}]},{"osd":3,"last update":"Fri Jun 14 22:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.439,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73399999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.51100000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.51900000000000002}]},{"osd":4,"last update":"Fri Jun 14 22:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.39200000000000002,"15min":0.379},"min":{"1min":0.313,"5min":0.248,"15min":0.248},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.40400000000000003,"15min":0.39400000000000002},"min":{"1min":0.34399999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.34000000000000002}]},{"osd":6,"last update":"Fri Jun 14 22:37:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.29299999999999998}]},{"osd":7,"last update":"Fri Jun 14 22:37:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.315}]}]},{"osd":4,"up_from":31,"seq":133143986210,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297976,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447176,"statfs":{"total":95995035648,"available":95689908224,"internally_reserved":0,"allocated":634880,"data_stored":265861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:37:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.57299999999999995,"15min":0.58299999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.749,"5min":1.026,"15min":1.026},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79300000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.55500000000000005}]},{"osd":1,"last update":"Fri Jun 14 22:37:47 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.52000000000000002,"15min":0.52600000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72999999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.63,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.53700000000000003}]},{"osd":2,"last update":"Fri Jun 14 22:37:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71799999999999997,"5min":0.873,"15min":0.873},"last":0.52400000000000002}]},{"osd":3,"last update":"Fri Jun 14 22:37:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57899999999999996,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76300000000000001,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.752,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.39800000000000002}]},{"osd":5,"last update":"Fri Jun 14 22:38:13 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.37,"15min":0.35999999999999999},"min":{"1min":0.29299999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.377,"15min":0.371},"min":{"1min":0.30099999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.40899999999999997}]},{"osd":6,"last update":"Fri Jun 14 22:37:37 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.504}]},{"osd":7,"last update":"Fri Jun 14 22:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.44400000000000001}]}]},{"osd":3,"up_from":26,"seq":111669149735,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297980,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447172,"statfs":{"total":95995035648,"available":95689904128,"internally_reserved":0,"allocated":638976,"data_stored":266126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:37:28 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.40100000000000002,"15min":0.40300000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.433,"15min":0.433},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.59699999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.38800000000000001}]},{"osd":1,"last update":"Fri Jun 14 22:37:28 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.33600000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.503},{"interface":"front","average":{"1min":0.434,"5min":0.42199999999999999,"15min":0.41999999999999998},"min":{"1min":0.34000000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.57299999999999995,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.48799999999999999}]},{"osd":2,"last update":"Fri Jun 14 22:37:28 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.442},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.44,"15min":0.437},"min":{"1min":0.317,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.45800000000000002}]},{"osd":4,"last update":"Fri Jun 14 22:37:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.56100000000000005}]},{"osd":5,"last update":"Fri Jun 14 22:38:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.70099999999999996,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.626}]},{"osd":6,"last update":"Fri Jun 14 22:37:35 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.61399999999999999}]},{"osd":7,"last update":"Fri Jun 14 22:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.58999999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411372,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297980,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447172,"statfs":{"total":95995035648,"available":95689904128,"internally_reserved":0,"allocated":638976,"data_stored":266126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.38500000000000001,"15min":0.38300000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.52100000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.39200000000000002,"15min":0.39300000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.45400000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.317}]},{"osd":1,"last update":"Fri Jun 14 22:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39200000000000002,"15min":0.38700000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.52600000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.245},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.28699999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.54500000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.30299999999999999}]},{"osd":3,"last update":"Fri Jun 14 22:37:32 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.45800000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.46800000000000003,"15min":0.47099999999999997},"min":{"1min":0.32000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.38400000000000001}]},{"osd":4,"last update":"Fri Jun 14 22:37:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61499999999999999,"15min":0.60999999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85999999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58599999999999997,"15min":0.58099999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.441}]},{"osd":5,"last update":"Fri Jun 14 22:38:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.84599999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61499999999999999,"15min":0.61899999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79200000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.42799999999999999}]},{"osd":6,"last update":"Fri Jun 14 22:37:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54700000000000004}]},{"osd":7,"last update":"Fri Jun 14 22:37:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.496},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.56399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297980,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447172,"statfs":{"total":95995035648,"available":95689904128,"internally_reserved":0,"allocated":638976,"data_stored":266126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:37:37 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.34499999999999997,"15min":0.33200000000000002},"min":{"1min":0.219,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.56999999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.35999999999999999,"15min":0.34399999999999997},"min":{"1min":0.248,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.32300000000000001}]},{"osd":2,"last update":"Fri Jun 14 22:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42399999999999999,"15min":0.41799999999999998},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.65200000000000002,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.38900000000000001,"15min":0.39000000000000001},"min":{"1min":0.248,"5min":0.246,"15min":0.246},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.309}]},{"osd":3,"last update":"Fri Jun 14 22:37:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.41299999999999998,"15min":0.41099999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61699999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.46899999999999997,"15min":0.46500000000000002},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.36899999999999999}]},{"osd":4,"last update":"Fri Jun 14 22:37:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.55100000000000005},"min":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.622}]},{"osd":5,"last update":"Fri Jun 14 22:38:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60899999999999999,"15min":0.61399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.80000000000000004,"5min":1.032,"15min":1.032},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.45100000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74299999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.68300000000000005}]},{"osd":6,"last update":"Fri Jun 14 22:37:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Jun 14 22:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.65900000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738422,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298552,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446600,"statfs":{"total":95995035648,"available":95689318400,"internally_reserved":0,"allocated":1224704,"data_stored":855964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 14 22:37:40 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.31},"min":{"1min":0.26500000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.59899999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.32200000000000001},"min":{"1min":0.26400000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.56000000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.35699999999999998}]},{"osd":2,"last update":"Fri Jun 14 22:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.40200000000000002,"15min":0.375},"min":{"1min":0.255,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.38600000000000001,"15min":0.36199999999999999},"min":{"1min":0.26300000000000001,"5min":0.248,"15min":0.248},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.376}]},{"osd":3,"last update":"Fri Jun 14 22:37:25 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.41699999999999998,"15min":0.42099999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48299999999999998,"15min":0.48499999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.61399999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.40400000000000003}]},{"osd":4,"last update":"Fri Jun 14 22:37:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.74099999999999999,"5min":1.048,"15min":1.048},"last":0.501},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93400000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.48299999999999998}]},{"osd":5,"last update":"Fri Jun 14 22:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77600000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.496},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.53100000000000003}]},{"osd":6,"last update":"Fri Jun 14 22:37:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.51200000000000001}]},{"osd":7,"last update":"Fri Jun 14 22:37:58 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.46600000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-14T22:38:30.710 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph pg dump --format=json 2024-06-14T22:38:30.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:30 smithi178 ceph-mon[32942]: Reconfiguring alertmanager.a (dependencies changed)... 2024-06-14T22:38:30.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:30 smithi178 ceph-mon[32942]: Reconfiguring daemon alertmanager.a on smithi046 2024-06-14T22:38:30.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:30 smithi178 ceph-mon[32942]: from='client.14583 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-14T22:38:30.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:30 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:38:31.357 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:31 smithi046 ceph-mon[29586]: pgmap v36: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:31.358 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:31 smithi046 ceph-mon[38286]: pgmap v36: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:31.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:31 smithi178 ceph-mon[32942]: pgmap v36: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:31.965 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:38:32.943 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:32 smithi046 systemd[1]: Stopping Ceph alertmanager.a for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:38:33.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:33 smithi046 ceph-mon[38286]: pgmap v37: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:33.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:33 smithi046 ceph-mon[29586]: pgmap v37: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:33.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:33 smithi178 ceph-mon[32942]: pgmap v37: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:33.927 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:33 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[69455]: ts=2024-06-14T22:38:33.564Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-06-14T22:38:33.927 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:33 smithi046 podman[74242]: 2024-06-14 22:38:33.669726009 +0000 UTC m=+0.232720443 container died d72e10fbe29210e219f855edafbff3e1c1590090ea7cb915d560c13f03857dab (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-14T22:38:34.192 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:34 smithi046 podman[74242]: 2024-06-14 22:38:34.021053176 +0000 UTC m=+0.584047602 container cleanup d72e10fbe29210e219f855edafbff3e1c1590090ea7cb915d560c13f03857dab (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-14T22:38:34.193 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:34 smithi046 bash[74242]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a 2024-06-14T22:38:34.524 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:34 smithi046 podman[74273]: 2024-06-14 22:38:34.24739955 +0000 UTC m=+0.626925533 container remove d72e10fbe29210e219f855edafbff3e1c1590090ea7cb915d560c13f03857dab (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-14T22:38:34.524 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:34 smithi046 podman[74273]: 2024-06-14 22:38:34.314733604 +0000 UTC m=+0.694259515 volume remove 7b54f6a7c442dc4233077384890110625eea18ac1e2242e9bba1d4c5a98d315f 2024-06-14T22:38:34.910 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:38:34.910 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-06-14T22:38:35.193 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:34 smithi046 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-06-14T22:38:35.193 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:34 smithi046 systemd[1]: Stopped Ceph alertmanager.a for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:38:35.193 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:34 smithi046 systemd[1]: Starting Ceph alertmanager.a for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:38:35.592 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:35 smithi046 ceph-mon[29586]: pgmap v38: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:35.592 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:35 smithi046 ceph-mon[38286]: pgmap v38: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:35.617 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-06-14T22:38:34.210514+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":27,"acting":27,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":27,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385548,"kb_used_data":6700,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575668,"statfs":{"total":767960285184,"available":765517484032,"internally_reserved":0,"allocated":6860800,"data_stored":3897876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001883"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-06-14T22:37:32.230249+0000","last_change":"2024-06-14T22:37:30.196467+0000","last_active":"2024-06-14T22:37:32.230249+0000","last_peered":"2024-06-14T22:37:32.230249+0000","last_clean":"2024-06-14T22:37:32.230249+0000","last_became_active":"2024-06-14T22:37:30.196165+0000","last_became_peered":"2024-06-14T22:37:30.196165+0000","last_unstale":"2024-06-14T22:37:32.230249+0000","last_undegraded":"2024-06-14T22:37:32.230249+0000","last_fullsized":"2024-06-14T22:37:32.230249+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:27:43.756983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-06-14T22:37:32.586721+0000","last_change":"2024-06-14T22:37:30.198386+0000","last_active":"2024-06-14T22:37:32.586721+0000","last_peered":"2024-06-14T22:37:32.586721+0000","last_clean":"2024-06-14T22:37:32.586721+0000","last_became_active":"2024-06-14T22:37:30.198073+0000","last_became_peered":"2024-06-14T22:37:30.198073+0000","last_unstale":"2024-06-14T22:37:32.586721+0000","last_undegraded":"2024-06-14T22:37:32.586721+0000","last_fullsized":"2024-06-14T22:37:32.586721+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T00:16:05.703306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-06-14T22:37:32.204596+0000","last_change":"2024-06-14T22:37:30.196859+0000","last_active":"2024-06-14T22:37:32.204596+0000","last_peered":"2024-06-14T22:37:32.204596+0000","last_clean":"2024-06-14T22:37:32.204596+0000","last_became_active":"2024-06-14T22:37:30.196536+0000","last_became_peered":"2024-06-14T22:37:30.196536+0000","last_unstale":"2024-06-14T22:37:32.204596+0000","last_undegraded":"2024-06-14T22:37:32.204596+0000","last_fullsized":"2024-06-14T22:37:32.204596+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:46:59.482280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-06-14T22:37:32.586717+0000","last_change":"2024-06-14T22:37:30.197518+0000","last_active":"2024-06-14T22:37:32.586717+0000","last_peered":"2024-06-14T22:37:32.586717+0000","last_clean":"2024-06-14T22:37:32.586717+0000","last_became_active":"2024-06-14T22:37:30.197224+0000","last_became_peered":"2024-06-14T22:37:30.197224+0000","last_unstale":"2024-06-14T22:37:32.586717+0000","last_undegraded":"2024-06-14T22:37:32.586717+0000","last_fullsized":"2024-06-14T22:37:32.586717+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T03:41:54.860308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-06-14T22:37:32.586955+0000","last_change":"2024-06-14T22:37:31.471372+0000","last_active":"2024-06-14T22:37:32.586955+0000","last_peered":"2024-06-14T22:37:32.586955+0000","last_clean":"2024-06-14T22:37:32.586955+0000","last_became_active":"2024-06-14T22:37:31.471087+0000","last_became_peered":"2024-06-14T22:37:31.471087+0000","last_unstale":"2024-06-14T22:37:32.586955+0000","last_undegraded":"2024-06-14T22:37:32.586955+0000","last_fullsized":"2024-06-14T22:37:32.586955+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:31:03.406918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-06-14T22:37:32.204209+0000","last_change":"2024-06-14T22:37:30.327693+0000","last_active":"2024-06-14T22:37:32.204209+0000","last_peered":"2024-06-14T22:37:32.204209+0000","last_clean":"2024-06-14T22:37:32.204209+0000","last_became_active":"2024-06-14T22:37:30.327313+0000","last_became_peered":"2024-06-14T22:37:30.327313+0000","last_unstale":"2024-06-14T22:37:32.204209+0000","last_undegraded":"2024-06-14T22:37:32.204209+0000","last_fullsized":"2024-06-14T22:37:32.204209+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T00:09:29.434302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-06-14T22:37:32.204499+0000","last_change":"2024-06-14T22:37:30.199225+0000","last_active":"2024-06-14T22:37:32.204499+0000","last_peered":"2024-06-14T22:37:32.204499+0000","last_clean":"2024-06-14T22:37:32.204499+0000","last_became_active":"2024-06-14T22:37:30.199011+0000","last_became_peered":"2024-06-14T22:37:30.199011+0000","last_unstale":"2024-06-14T22:37:32.204499+0000","last_undegraded":"2024-06-14T22:37:32.204499+0000","last_fullsized":"2024-06-14T22:37:32.204499+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:45:43.859896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-06-14T22:37:32.586955+0000","last_change":"2024-06-14T22:37:31.278821+0000","last_active":"2024-06-14T22:37:32.586955+0000","last_peered":"2024-06-14T22:37:32.586955+0000","last_clean":"2024-06-14T22:37:32.586955+0000","last_became_active":"2024-06-14T22:37:31.278504+0000","last_became_peered":"2024-06-14T22:37:31.278504+0000","last_unstale":"2024-06-14T22:37:32.586955+0000","last_undegraded":"2024-06-14T22:37:32.586955+0000","last_fullsized":"2024-06-14T22:37:32.586955+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T10:00:23.519181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"63'128","reported_seq":271,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-06-14T22:37:32.255867+0000","last_change":"2024-06-14T22:37:01.481768+0000","last_active":"2024-06-14T22:37:32.255867+0000","last_peered":"2024-06-14T22:37:32.255867+0000","last_clean":"2024-06-14T22:37:32.255867+0000","last_became_active":"2024-06-14T22:37:01.481339+0000","last_became_peered":"2024-06-14T22:37:01.481339+0000","last_unstale":"2024-06-14T22:37:32.255867+0000","last_undegraded":"2024-06-14T22:37:32.255867+0000","last_fullsized":"2024-06-14T22:37:32.255867+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:34:59.252736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:34:59.252736+0000","last_clean_scrub_stamp":"2024-06-14T22:34:59.252736+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T22:55:57.126904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201301,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298552,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446600,"statfs":{"total":95995035648,"available":95689318400,"internally_reserved":0,"allocated":1224704,"data_stored":855906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Jun 14 22:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.5,"15min":0.5},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.60599999999999998}]},{"osd":2,"last update":"Fri Jun 14 22:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.438}]},{"osd":3,"last update":"Fri Jun 14 22:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.47799999999999998}]},{"osd":4,"last update":"Fri Jun 14 22:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.37}]},{"osd":5,"last update":"Fri Jun 14 22:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.38900000000000001}]},{"osd":6,"last update":"Fri Jun 14 22:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55500000000000005}]}]},{"osd":6,"up_from":47,"seq":201863462939,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298548,"kb_used_data":1192,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446604,"statfs":{"total":95995035648,"available":95689322496,"internally_reserved":0,"allocated":1220608,"data_stored":855641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:37:35 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.51000000000000001}]},{"osd":1,"last update":"Fri Jun 14 22:37:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.47599999999999998}]},{"osd":2,"last update":"Fri Jun 14 22:37:35 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.56499999999999995}]},{"osd":3,"last update":"Fri Jun 14 22:37:35 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.58699999999999997}]},{"osd":4,"last update":"Fri Jun 14 22:37:35 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.44700000000000001}]},{"osd":5,"last update":"Fri Jun 14 22:37:35 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.379}]},{"osd":7,"last update":"Fri Jun 14 22:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.49299999999999999}]}]},{"osd":5,"up_from":39,"seq":167503724575,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297980,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447172,"statfs":{"total":95995035648,"available":95689904128,"internally_reserved":0,"allocated":638976,"data_stored":266126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.379,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.51800000000000002,"15min":0.51000000000000001},"min":{"1min":0.378,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.78900000000000003,"5min":0.873,"15min":0.873},"last":0.63900000000000001}]},{"osd":1,"last update":"Fri Jun 14 22:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.40699999999999997,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.70499999999999996}]},{"osd":2,"last update":"Fri Jun 14 22:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.51500000000000001,"15min":0.50600000000000001},"min":{"1min":0.39100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.751},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.434,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.56599999999999995}]},{"osd":3,"last update":"Fri Jun 14 22:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.53500000000000003,"15min":0.52900000000000003},"min":{"1min":0.439,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73399999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.54800000000000004,"15min":0.54000000000000004},"min":{"1min":0.51100000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.60699999999999998}]},{"osd":4,"last update":"Fri Jun 14 22:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.39200000000000002,"15min":0.379},"min":{"1min":0.313,"5min":0.248,"15min":0.248},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.40400000000000003,"15min":0.39400000000000002},"min":{"1min":0.34399999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.41399999999999998}]},{"osd":6,"last update":"Fri Jun 14 22:37:34 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.48099999999999998}]},{"osd":7,"last update":"Fri Jun 14 22:37:59 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.438},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.46300000000000002}]}]},{"osd":4,"up_from":31,"seq":133143986211,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297976,"kb_used_data":620,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447176,"statfs":{"total":95995035648,"available":95689908224,"internally_reserved":0,"allocated":634880,"data_stored":265861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:37:47 2024","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.57299999999999995,"15min":0.58299999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.749,"5min":1.026,"15min":1.026},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.79300000000000004,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Jun 14 22:37:47 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.52000000000000002,"15min":0.52600000000000002},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.72999999999999998,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.57799999999999996,"15min":0.58999999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.63,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.57199999999999995}]},{"osd":2,"last update":"Fri Jun 14 22:37:47 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.624},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.54900000000000004,"15min":0.55500000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.71799999999999997,"5min":0.873,"15min":0.873},"last":0.60499999999999998}]},{"osd":3,"last update":"Fri Jun 14 22:37:47 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.57899999999999996,"15min":0.58399999999999996},"min":{"1min":0.40100000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.76300000000000001,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.752,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.56200000000000006}]},{"osd":5,"last update":"Fri Jun 14 22:38:13 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.37,"15min":0.35999999999999999},"min":{"1min":0.29299999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.377,"15min":0.371},"min":{"1min":0.30099999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.48299999999999998}]},{"osd":6,"last update":"Fri Jun 14 22:37:37 2024","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.46400000000000002}]},{"osd":7,"last update":"Fri Jun 14 22:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.5}]}]},{"osd":3,"up_from":26,"seq":111669149736,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297980,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447172,"statfs":{"total":95995035648,"available":95689904128,"internally_reserved":0,"allocated":638976,"data_stored":266126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.39400000000000002,"15min":0.40000000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.41799999999999998,"15min":0.42799999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.22800000000000001}]},{"osd":1,"last update":"Fri Jun 14 22:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.42099999999999999,"15min":0.41799999999999998},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.41699999999999998,"15min":0.41899999999999998},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.61399999999999999,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.26800000000000002}]},{"osd":2,"last update":"Fri Jun 14 22:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.44400000000000001,"15min":0.45200000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.432,"15min":0.434},"min":{"1min":0.27800000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.251}]},{"osd":4,"last update":"Fri Jun 14 22:37:51 2024","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.56799999999999995,"15min":0.56399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.66300000000000003}]},{"osd":5,"last update":"Fri Jun 14 22:38:14 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57299999999999995,"15min":0.58099999999999996},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.70099999999999996,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.497},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.73199999999999998}]},{"osd":6,"last update":"Fri Jun 14 22:37:35 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54200000000000004}]},{"osd":7,"last update":"Fri Jun 14 22:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.76100000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297980,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447172,"statfs":{"total":95995035648,"available":95689904128,"internally_reserved":0,"allocated":638976,"data_stored":266126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.38500000000000001,"15min":0.38300000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.52100000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.39200000000000002,"15min":0.39300000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.45400000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.46000000000000002}]},{"osd":1,"last update":"Fri Jun 14 22:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39200000000000002,"15min":0.38700000000000001},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.52600000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.499},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.40200000000000002,"15min":0.39600000000000002},"min":{"1min":0.28699999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.54500000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.628}]},{"osd":3,"last update":"Fri Jun 14 22:37:32 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.45800000000000002},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.46800000000000003,"15min":0.47099999999999997},"min":{"1min":0.32000000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.55200000000000005}]},{"osd":4,"last update":"Fri Jun 14 22:37:45 2024","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61499999999999999,"15min":0.60999999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.85999999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.58599999999999997,"15min":0.58099999999999996},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.61599999999999999}]},{"osd":5,"last update":"Fri Jun 14 22:38:11 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60299999999999998,"15min":0.60999999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.84599999999999997,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61499999999999999,"15min":0.61899999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79200000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.60699999999999998}]},{"osd":6,"last update":"Fri Jun 14 22:37:34 2024","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.73299999999999998}]},{"osd":7,"last update":"Fri Jun 14 22:37:58 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64600000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297980,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447172,"statfs":{"total":95995035648,"available":95689904128,"internally_reserved":0,"allocated":638976,"data_stored":266126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:37:37 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.34499999999999997,"15min":0.33200000000000002},"min":{"1min":0.219,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.56999999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.214},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.35999999999999999,"15min":0.34399999999999997},"min":{"1min":0.248,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.23100000000000001}]},{"osd":2,"last update":"Fri Jun 14 22:38:03 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42399999999999999,"15min":0.41799999999999998},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.65200000000000002,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.38900000000000001,"15min":0.39000000000000001},"min":{"1min":0.248,"5min":0.246,"15min":0.246},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.252}]},{"osd":3,"last update":"Fri Jun 14 22:38:29 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.40699999999999997,"15min":0.40899999999999997},"min":{"1min":0.28100000000000003,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.52500000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.46200000000000002,"15min":0.46300000000000002},"min":{"1min":0.311,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.59499999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.32400000000000001}]},{"osd":4,"last update":"Fri Jun 14 22:37:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.55100000000000005},"min":{"1min":0.41799999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.59899999999999998}]},{"osd":5,"last update":"Fri Jun 14 22:38:14 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60899999999999999,"15min":0.61399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.80000000000000004,"5min":1.032,"15min":1.032},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.59099999999999997,"15min":0.59399999999999997},"min":{"1min":0.45100000000000001,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.74299999999999999,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.58899999999999997}]},{"osd":6,"last update":"Fri Jun 14 22:37:33 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.55100000000000005}]},{"osd":7,"last update":"Fri Jun 14 22:38:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.63100000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298552,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446600,"statfs":{"total":95995035648,"available":95689318400,"internally_reserved":0,"allocated":1224704,"data_stored":855964,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 14 22:37:40 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.34300000000000003,"15min":0.31},"min":{"1min":0.26500000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.59899999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.34599999999999997,"15min":0.32200000000000001},"min":{"1min":0.26400000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.56000000000000005,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.30199999999999999}]},{"osd":2,"last update":"Fri Jun 14 22:38:02 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.40200000000000002,"15min":0.375},"min":{"1min":0.255,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.38600000000000001,"15min":0.36199999999999999},"min":{"1min":0.26300000000000001,"5min":0.248,"15min":0.248},"max":{"1min":1.51,"5min":1.51,"15min":1.51},"last":0.26900000000000002}]},{"osd":3,"last update":"Fri Jun 14 22:38:27 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41199999999999998,"15min":0.41999999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47999999999999998,"15min":0.48399999999999999},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.36299999999999999}]},{"osd":4,"last update":"Fri Jun 14 22:37:45 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.60899999999999999,"15min":0.61099999999999999},"min":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.74099999999999999,"5min":1.048,"15min":1.048},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.93400000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.44800000000000001}]},{"osd":5,"last update":"Fri Jun 14 22:38:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.57699999999999996,"15min":0.58199999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.77600000000000002,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.61099999999999999,"15min":0.60699999999999998},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.439}]},{"osd":6,"last update":"Fri Jun 14 22:37:37 2024","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.498},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.46700000000000003}]},{"osd":7,"last update":"Fri Jun 14 22:37:58 2024","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.673,"5min":1.673,"15min":1.673},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.51100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-14T22:38:35.619 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-14T22:38:35.619 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-14T22:38:35.619 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-14T22:38:35.619 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph health --format=json 2024-06-14T22:38:35.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:35 smithi178 ceph-mon[32942]: pgmap v38: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:35.876 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:35 smithi046 podman[74451]: 2024-06-14 22:38:35.78864927 +0000 UTC m=+0.105513746 volume create e4762026dba5c496f544dfc64e74fa037f9168e3a4a3efb9237dbd4dca02eba4 2024-06-14T22:38:35.877 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:35 smithi046 podman[74451]: 2024-06-14 22:38:35.701393038 +0000 UTC m=+0.018257531 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-06-14T22:38:35.877 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:35 smithi046 podman[74451]: 2024-06-14 22:38:35.880841981 +0000 UTC m=+0.197706460 container create 746b52e70c667d778450660efb38a31dd12c7046dc2598cccf323a91108ac0a4 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-14T22:38:36.420 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:38:36.544 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:36 smithi046 ceph-mon[29586]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-14T22:38:36.545 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:36 smithi046 ceph-mon[38286]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-14T22:38:36.545 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:36 smithi046 podman[74451]: 2024-06-14 22:38:36.200301156 +0000 UTC m=+0.517165644 container init 746b52e70c667d778450660efb38a31dd12c7046dc2598cccf323a91108ac0a4 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-14T22:38:36.545 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:36 smithi046 podman[74451]: 2024-06-14 22:38:36.206603388 +0000 UTC m=+0.523467865 container start 746b52e70c667d778450660efb38a31dd12c7046dc2598cccf323a91108ac0a4 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-14T22:38:36.545 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:36 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[74477]: ts=2024-06-14T22:38:36.219Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-06-14T22:38:36.545 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:36 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[74477]: ts=2024-06-14T22:38:36.219Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-06-14T22:38:36.545 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:36 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[74477]: ts=2024-06-14T22:38:36.220Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.46 port=9094 2024-06-14T22:38:36.545 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:36 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[74477]: ts=2024-06-14T22:38:36.221Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-06-14T22:38:36.546 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:36 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[74477]: ts=2024-06-14T22:38:36.244Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-14T22:38:36.546 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:36 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[74477]: ts=2024-06-14T22:38:36.244Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-06-14T22:38:36.546 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:36 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[74477]: ts=2024-06-14T22:38:36.247Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-06-14T22:38:36.546 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:36 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[74477]: ts=2024-06-14T22:38:36.247Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-06-14T22:38:36.546 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:36 smithi046 bash[74451]: 746b52e70c667d778450660efb38a31dd12c7046dc2598cccf323a91108ac0a4 2024-06-14T22:38:36.546 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:36 smithi046 systemd[1]: Started Ceph alertmanager.a for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:38:36.609 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:36 smithi178 ceph-mon[32942]: from='client.14589 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-14T22:38:37.603 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:37 smithi046 ceph-mon[38286]: pgmap v39: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:37.603 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:37 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:37.603 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:37 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:37.603 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:37 smithi046 ceph-mon[38286]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-14T22:38:37.604 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:37 smithi046 ceph-mon[38286]: Reconfiguring daemon prometheus.a on smithi178 2024-06-14T22:38:37.604 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:37 smithi046 ceph-mon[29586]: pgmap v39: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:37.604 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:37 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:37.604 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:37 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:37.604 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:37 smithi046 ceph-mon[29586]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-14T22:38:37.604 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:37 smithi046 ceph-mon[29586]: Reconfiguring daemon prometheus.a on smithi178 2024-06-14T22:38:37.775 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:38:37 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:38:37] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-06-14T22:38:37.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:37 smithi178 ceph-mon[32942]: pgmap v39: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:37.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:37 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:37.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:37 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:37.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:37 smithi178 ceph-mon[32942]: Reconfiguring prometheus.a (dependencies changed)... 2024-06-14T22:38:37.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:37 smithi178 ceph-mon[32942]: Reconfiguring daemon prometheus.a on smithi178 2024-06-14T22:38:38.335 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:38:38.335 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-14T22:38:38.627 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:38 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[74477]: ts=2024-06-14T22:38:38.222Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000995788s 2024-06-14T22:38:39.003 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-14T22:38:39.003 INFO:tasks.cephadm:Setup complete, yielding 2024-06-14T22:38:39.003 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-14T22:38:39.012 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-06-14T22:38:39.013 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- bash -c 'ceph fs volume create foo' 2024-06-14T22:38:39.276 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:38:39.525 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:39 smithi178 systemd[1]: Stopping Ceph prometheus.a for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:38:39.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:39 smithi178 ceph-mon[32942]: pgmap v40: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:39.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:39 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/4288043041' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-14T22:38:39.655 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:39 smithi046 ceph-mon[29586]: pgmap v40: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:39.655 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:39 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/4288043041' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-14T22:38:39.656 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:39 smithi046 ceph-mon[38286]: pgmap v40: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:39.656 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:39 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/4288043041' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-14T22:38:40.026 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:39 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:38:39.675Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-06-14T22:38:40.026 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:39 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:38:39.675Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-06-14T22:38:40.026 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:39 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:38:39.675Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-06-14T22:38:40.026 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:39 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:38:39.675Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-06-14T22:38:40.026 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:39 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:38:39.675Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-06-14T22:38:40.026 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:39 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:38:39.675Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-06-14T22:38:40.026 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:39 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:38:39.675Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-06-14T22:38:40.027 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:39 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:38:39.675Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-06-14T22:38:40.027 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:39 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:38:39.675Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-06-14T22:38:40.027 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:39 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:38:39.675Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-06-14T22:38:40.027 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:39 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:38:39.675Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-06-14T22:38:40.027 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:39 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[61616]: ts=2024-06-14T22:38:39.675Z caller=main.go:1147 level=info msg="See you next time!" 2024-06-14T22:38:40.027 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:39 smithi178 podman[64892]: 2024-06-14 22:38:39.685581301 +0000 UTC m=+0.121665483 container died 5ebccc451606087474e4b8e9b46cd83cefd32f5c3a1a61e2371e334f80d66a3a (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-14T22:38:40.523 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:40 smithi178 podman[64892]: 2024-06-14 22:38:40.12167586 +0000 UTC m=+0.557760041 container cleanup 5ebccc451606087474e4b8e9b46cd83cefd32f5c3a1a61e2371e334f80d66a3a (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-14T22:38:40.523 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:40 smithi178 bash[64892]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a 2024-06-14T22:38:40.523 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:40 smithi178 podman[64912]: 2024-06-14 22:38:40.289823097 +0000 UTC m=+0.603150568 container remove 5ebccc451606087474e4b8e9b46cd83cefd32f5c3a1a61e2371e334f80d66a3a (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-14T22:38:40.794 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:40 smithi178 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-06-14T22:38:40.794 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:40 smithi178 systemd[1]: Stopped Ceph prometheus.a for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:38:40.795 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:40 smithi178 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@prometheus.a.service: Consumed 1.120s CPU time. 2024-06-14T22:38:40.795 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:40 smithi178 systemd[1]: Starting Ceph prometheus.a for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:38:40.795 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:40 smithi178 podman[65008]: 2024-06-14 22:38:40.660144676 +0000 UTC m=+0.017985613 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-06-14T22:38:41.090 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:40 smithi178 podman[65008]: 2024-06-14 22:38:40.798951325 +0000 UTC m=+0.156792260 container create cdc6069291089375981a835f2255eb57a74f00cc3aefab89b4990cf1e5c76151 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-14T22:38:41.090 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:40 smithi178 podman[65008]: 2024-06-14 22:38:40.992531377 +0000 UTC m=+0.350372313 container init cdc6069291089375981a835f2255eb57a74f00cc3aefab89b4990cf1e5c76151 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-14T22:38:41.090 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:40 smithi178 podman[65008]: 2024-06-14 22:38:40.99831138 +0000 UTC m=+0.356152319 container start cdc6069291089375981a835f2255eb57a74f00cc3aefab89b4990cf1e5c76151 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-14T22:38:41.090 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.022Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-06-14T22:38:41.091 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.022Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-06-14T22:38:41.091 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.022Z caller=main.go:570 level=info host_details="(Linux 5.14.0-457.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu May 30 22:48:58 UTC 2024 x86_64 smithi178 (none))" 2024-06-14T22:38:41.091 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.022Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-06-14T22:38:41.091 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.022Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-06-14T22:38:41.091 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.023Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-06-14T22:38:41.091 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.024Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-06-14T22:38:41.091 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.025Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-06-14T22:38:41.091 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.025Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-06-14T22:38:41.091 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.027Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-06-14T22:38:41.091 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.027Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.706µs 2024-06-14T22:38:41.091 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.027Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-06-14T22:38:41.092 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.027Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-06-14T22:38:41.092 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.027Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-06-14T22:38:41.092 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.027Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=19.239µs wal_replay_duration=394.908µs wbl_replay_duration=122ns total_replay_duration=431.69µs 2024-06-14T22:38:41.092 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.028Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-06-14T22:38:41.092 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.028Z caller=main.go:1029 level=info msg="TSDB started" 2024-06-14T22:38:41.092 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.028Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-06-14T22:38:41.092 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.044Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.857172ms db_storage=688ns remote_storage=1.116µs web_handler=365ns query_engine=452ns scrape=437.187µs scrape_sd=24.867µs notify=44.081µs notify_sd=5.825µs rules=15.135756ms tracing=6.067µs 2024-06-14T22:38:41.092 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.044Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-06-14T22:38:41.092 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:38:41.044Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-06-14T22:38:41.092 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 bash[65008]: cdc6069291089375981a835f2255eb57a74f00cc3aefab89b4990cf1e5c76151 2024-06-14T22:38:41.412 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:38:41 smithi178 systemd[1]: Started Ceph prometheus.a for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:38:41.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:41 smithi046 ceph-mon[29586]: pgmap v41: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:41.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:41 smithi046 ceph-mon[38286]: pgmap v41: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:41.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:38:41 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:38:41] ENGINE Bus STOPPING 2024-06-14T22:38:41.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:38:41 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:38:41] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-14T22:38:41.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:38:41 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:38:41] ENGINE Bus STOPPED 2024-06-14T22:38:41.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:38:41 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:38:41] ENGINE Bus STARTING 2024-06-14T22:38:41.694 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:38:41 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:38:41] ENGINE Serving on http://:::9283 2024-06-14T22:38:41.694 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:38:41 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:38:41] ENGINE Bus STARTED 2024-06-14T22:38:41.694 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:38:41 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:38:41] ENGINE Bus STOPPING 2024-06-14T22:38:41.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:41 smithi178 ceph-mon[32942]: pgmap v41: 9 pgs: 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:42.413 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:38:42] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-14T22:38:42.413 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:38:42] ENGINE Bus STOPPED 2024-06-14T22:38:42.413 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:38:42] ENGINE Bus STARTING 2024-06-14T22:38:42.413 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:38:42] ENGINE Serving on http://:::9283 2024-06-14T22:38:42.413 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:38:42] ENGINE Bus STARTED 2024-06-14T22:38:42.414 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:38:42] ENGINE Bus STOPPING 2024-06-14T22:38:42.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='client.24497 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "foo", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:38:42.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd pool create", "pool": "cephfs.foo.meta"}]: dispatch 2024-06-14T22:38:42.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:42.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:42.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-14T22:38:42.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-14T22:38:42.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi046.front.sepia.ceph.com:9093"}]: dispatch 2024-06-14T22:38:42.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi046.front.sepia.ceph.com:9093"}]: dispatch 2024-06-14T22:38:42.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:42.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-14T22:38:42.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-14T22:38:42.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi178.front.sepia.ceph.com:9095"}]: dispatch 2024-06-14T22:38:42.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi178.front.sepia.ceph.com:9095"}]: dispatch 2024-06-14T22:38:42.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:42.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-14T22:38:42.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-14T22:38:42.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi178.front.sepia.ceph.com:3000"}]: dispatch 2024-06-14T22:38:42.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi178.front.sepia.ceph.com:3000"}]: dispatch 2024-06-14T22:38:42.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:42.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:38:42.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "osd pool create", "pool": "cephfs.foo.meta"}]': finished 2024-06-14T22:38:42.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: osdmap e65: 8 total, 8 up, 8 in 2024-06-14T22:38:42.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.foo.data"}]: dispatch 2024-06-14T22:38:42.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='client.24497 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "foo", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:38:42.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd pool create", "pool": "cephfs.foo.meta"}]: dispatch 2024-06-14T22:38:42.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:42.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:42.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-14T22:38:42.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-14T22:38:42.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi046.front.sepia.ceph.com:9093"}]: dispatch 2024-06-14T22:38:42.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi046.front.sepia.ceph.com:9093"}]: dispatch 2024-06-14T22:38:42.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:42.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-14T22:38:42.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-14T22:38:42.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi178.front.sepia.ceph.com:9095"}]: dispatch 2024-06-14T22:38:42.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi178.front.sepia.ceph.com:9095"}]: dispatch 2024-06-14T22:38:42.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:42.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-14T22:38:42.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-14T22:38:42.697 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi178.front.sepia.ceph.com:3000"}]: dispatch 2024-06-14T22:38:42.697 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi178.front.sepia.ceph.com:3000"}]: dispatch 2024-06-14T22:38:42.697 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:42.697 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:38:42.697 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "osd pool create", "pool": "cephfs.foo.meta"}]': finished 2024-06-14T22:38:42.697 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: osdmap e65: 8 total, 8 up, 8 in 2024-06-14T22:38:42.697 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.foo.data"}]: dispatch 2024-06-14T22:38:42.697 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-a[29563]: 2024-06-14T22:38:42.479+0000 7f95b0ad6640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-14T22:38:42.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='client.24497 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "foo", "target": ["mon-mgr", ""]}]: dispatch 2024-06-14T22:38:42.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd pool create", "pool": "cephfs.foo.meta"}]: dispatch 2024-06-14T22:38:42.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:42.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:42.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-14T22:38:42.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-06-14T22:38:42.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi046.front.sepia.ceph.com:9093"}]: dispatch 2024-06-14T22:38:42.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi046.front.sepia.ceph.com:9093"}]: dispatch 2024-06-14T22:38:42.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:42.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-14T22:38:42.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-06-14T22:38:42.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi178.front.sepia.ceph.com:9095"}]: dispatch 2024-06-14T22:38:42.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi178.front.sepia.ceph.com:9095"}]: dispatch 2024-06-14T22:38:42.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:42.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-14T22:38:42.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-06-14T22:38:42.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi178.front.sepia.ceph.com:3000"}]: dispatch 2024-06-14T22:38:42.776 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi178.front.sepia.ceph.com:3000"}]: dispatch 2024-06-14T22:38:42.777 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:42.777 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:38:42.777 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "osd pool create", "pool": "cephfs.foo.meta"}]': finished 2024-06-14T22:38:42.777 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: osdmap e65: 8 total, 8 up, 8 in 2024-06-14T22:38:42.777 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:42 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.foo.data"}]: dispatch 2024-06-14T22:38:43.137 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:38:42] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-14T22:38:43.137 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:38:42] ENGINE Bus STOPPED 2024-06-14T22:38:43.137 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:38:42] ENGINE Bus STARTING 2024-06-14T22:38:43.137 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:38:42] ENGINE Serving on http://:::9283 2024-06-14T22:38:43.138 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:38:42 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:38:42] ENGINE Bus STARTED 2024-06-14T22:38:43.165 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-06-14T22:38:43.477 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:38:43.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[29586]: pgmap v43: 41 pgs: 32 unknown, 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:43.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[29586]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-14T22:38:43.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.foo.data"}]': finished 2024-06-14T22:38:43.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[29586]: osdmap e66: 8 total, 8 up, 8 in 2024-06-14T22:38:43.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "fs new", "fs_name": "foo", "metadata": "cephfs.foo.meta", "data": "cephfs.foo.data"}]: dispatch 2024-06-14T22:38:43.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[29586]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-14T22:38:43.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[29586]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-14T22:38:43.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[29586]: osdmap e67: 8 total, 8 up, 8 in 2024-06-14T22:38:43.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "fs new", "fs_name": "foo", "metadata": "cephfs.foo.meta", "data": "cephfs.foo.data"}]': finished 2024-06-14T22:38:43.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[29586]: fsmap foo:0 2024-06-14T22:38:43.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:43.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[38286]: pgmap v43: 41 pgs: 32 unknown, 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:43.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[38286]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-14T22:38:43.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.foo.data"}]': finished 2024-06-14T22:38:43.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[38286]: osdmap e66: 8 total, 8 up, 8 in 2024-06-14T22:38:43.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "fs new", "fs_name": "foo", "metadata": "cephfs.foo.meta", "data": "cephfs.foo.data"}]: dispatch 2024-06-14T22:38:43.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[38286]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-14T22:38:43.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[38286]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-14T22:38:43.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[38286]: osdmap e67: 8 total, 8 up, 8 in 2024-06-14T22:38:43.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "fs new", "fs_name": "foo", "metadata": "cephfs.foo.meta", "data": "cephfs.foo.data"}]': finished 2024-06-14T22:38:43.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[38286]: fsmap foo:0 2024-06-14T22:38:43.695 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:43 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:43.772 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:43 smithi178 ceph-mon[32942]: pgmap v43: 41 pgs: 32 unknown, 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:43.772 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:43 smithi178 ceph-mon[32942]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-14T22:38:43.773 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:43 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"bulk": true, "prefix": "osd pool create", "pool": "cephfs.foo.data"}]': finished 2024-06-14T22:38:43.773 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:43 smithi178 ceph-mon[32942]: osdmap e66: 8 total, 8 up, 8 in 2024-06-14T22:38:43.773 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:43 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "fs new", "fs_name": "foo", "metadata": "cephfs.foo.meta", "data": "cephfs.foo.data"}]: dispatch 2024-06-14T22:38:43.773 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:43 smithi178 ceph-mon[32942]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-14T22:38:43.773 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:43 smithi178 ceph-mon[32942]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-14T22:38:43.773 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:43 smithi178 ceph-mon[32942]: osdmap e67: 8 total, 8 up, 8 in 2024-06-14T22:38:43.773 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:43 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "fs new", "fs_name": "foo", "metadata": "cephfs.foo.meta", "data": "cephfs.foo.data"}]': finished 2024-06-14T22:38:43.773 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:43 smithi178 ceph-mon[32942]: fsmap foo:0 2024-06-14T22:38:43.773 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:43 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:44.570 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:44 smithi046 ceph-mon[29586]: Saving service mds.foo spec with placement count:2 2024-06-14T22:38:44.570 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:44 smithi046 ceph-mon[29586]: osdmap e68: 8 total, 8 up, 8 in 2024-06-14T22:38:44.571 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:44 smithi046 ceph-mon[38286]: Saving service mds.foo spec with placement count:2 2024-06-14T22:38:44.571 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:44 smithi046 ceph-mon[38286]: osdmap e68: 8 total, 8 up, 8 in 2024-06-14T22:38:44.768 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:44 smithi178 ceph-mon[32942]: Saving service mds.foo spec with placement count:2 2024-06-14T22:38:44.768 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:44 smithi178 ceph-mon[32942]: osdmap e68: 8 total, 8 up, 8 in 2024-06-14T22:38:45.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:45 smithi046 ceph-mon[38286]: pgmap v47: 73 pgs: 64 unknown, 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:45.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:45 smithi046 ceph-mon[38286]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-14T22:38:45.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:45 smithi046 ceph-mon[38286]: osdmap e69: 8 total, 8 up, 8 in 2024-06-14T22:38:45.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:45 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:45.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:45 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:45.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:45 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:45.694 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:45 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:38:45.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:45 smithi046 ceph-mon[29586]: pgmap v47: 73 pgs: 64 unknown, 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:45.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:45 smithi046 ceph-mon[29586]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-14T22:38:45.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:45 smithi046 ceph-mon[29586]: osdmap e69: 8 total, 8 up, 8 in 2024-06-14T22:38:45.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:45 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:45.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:45 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:45.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:45 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:45.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:45 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:38:45.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:45 smithi178 ceph-mon[32942]: pgmap v47: 73 pgs: 64 unknown, 9 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:45.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:45 smithi178 ceph-mon[32942]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-14T22:38:45.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:45 smithi178 ceph-mon[32942]: osdmap e69: 8 total, 8 up, 8 in 2024-06-14T22:38:45.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:45 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:45.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:45 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:45.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:45 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:45.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:45 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:38:46.443 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:38:46 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[74477]: ts=2024-06-14T22:38:46.223Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002354895s 2024-06-14T22:38:46.902 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-06-14T22:38:47.409 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:38:47.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:47 smithi046 ceph-mon[38286]: pgmap v49: 73 pgs: 73 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:47.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:47 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:47.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:47 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:47.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:47 smithi046 ceph-mon[29586]: pgmap v49: 73 pgs: 73 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:47.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:47 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:47.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:47 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:47.775 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:38:47 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:38:47] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-06-14T22:38:47.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:47 smithi178 ceph-mon[32942]: pgmap v49: 73 pgs: 73 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:47.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:47 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:47.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:47 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:48.609 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:48 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:38:48.609 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:48 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:38:48.609 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:48 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:48.609 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:48 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi046.fdgzyw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-14T22:38:48.609 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:48 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi046.fdgzyw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-14T22:38:48.609 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:48 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:38:48.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:48 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:38:48.609 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:48 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:38:48.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:48 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:48.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:48 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi046.fdgzyw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-14T22:38:48.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:48 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi046.fdgzyw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-14T22:38:48.610 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:48 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:38:48.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:48 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:38:48.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:48 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:38:48.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:48 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:48.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:48 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi046.fdgzyw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-14T22:38:48.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:48 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi046.fdgzyw", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-14T22:38:48.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:48 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:38:49.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:49 smithi046 ceph-mon[29586]: Deploying daemon mds.foo.smithi046.fdgzyw on smithi046 2024-06-14T22:38:49.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:49 smithi046 ceph-mon[29586]: pgmap v50: 73 pgs: 73 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:49.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:49 smithi046 ceph-mon[38286]: Deploying daemon mds.foo.smithi046.fdgzyw on smithi046 2024-06-14T22:38:49.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:49 smithi046 ceph-mon[38286]: pgmap v50: 73 pgs: 73 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:49.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:49 smithi178 ceph-mon[32942]: Deploying daemon mds.foo.smithi046.fdgzyw on smithi046 2024-06-14T22:38:49.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:49 smithi178 ceph-mon[32942]: pgmap v50: 73 pgs: 73 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:50.232 INFO:teuthology.run_tasks:Running task ceph.healthy... 2024-06-14T22:38:50.241 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-14T22:38:50.241 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-14T22:38:50.241 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph mgr dump --format=json 2024-06-14T22:38:50.845 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:38:51.681 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:51 smithi046 ceph-mon[29586]: pgmap v51: 73 pgs: 73 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:51.681 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:51 smithi046 ceph-mon[38286]: pgmap v51: 73 pgs: 73 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:51.682 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:38:51 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:38:51] "GET /metrics HTTP/1.1" 200 207929 "" "Prometheus/2.43.0" 2024-06-14T22:38:51.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:51 smithi178 ceph-mon[32942]: pgmap v51: 73 pgs: 73 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:52.932 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:38:53.517 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:53 smithi046 ceph-mon[38286]: pgmap v52: 73 pgs: 73 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:53.517 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:53 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/654370224' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-14T22:38:53.517 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:53 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:53.517 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:53 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:53.517 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:53 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:53.517 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:53 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi178.jvpymc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-14T22:38:53.517 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:53 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi178.jvpymc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-14T22:38:53.518 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:53 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:38:53.518 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:53 smithi046 ceph-mon[29586]: pgmap v52: 73 pgs: 73 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:53.518 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:53 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/654370224' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-14T22:38:53.518 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:53 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:53.518 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:53 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:53.518 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:53 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:53.519 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:53 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi178.jvpymc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-14T22:38:53.519 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:53 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi178.jvpymc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-14T22:38:53.519 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:53 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:38:53.543 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":21,"flags":0,"active_gid":14415,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":835295015},{"type":"v1","addr":"172.21.15.46:6801","nonce":835295015}]},"active_addr":"172.21.15.46:6801/835295015","active_change":"2024-06-14T22:37:30.185431+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24355,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":false,"error_string":"No module named 'kubernetes.client.models.v1_event'","module_options":{}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":false,"error_string":"No module named 'kubernetes.client.models.v1_event'","module_options":{}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.46:8443/","prometheus":"http://172.21.15.46:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":61,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":664260966}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":556015621}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":3331427503}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":245604168}]}]} 2024-06-14T22:38:53.547 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-14T22:38:53.547 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-14T22:38:53.547 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd dump --format=json 2024-06-14T22:38:53.672 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:53 smithi178 ceph-mon[32942]: pgmap v52: 73 pgs: 73 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:53.672 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:53 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/654370224' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-14T22:38:53.672 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:53 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:53.672 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:53 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:53.672 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:53 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:53.672 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:53 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi178.jvpymc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-14T22:38:53.672 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:53 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mds.foo.smithi178.jvpymc", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-14T22:38:53.672 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:53 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:38:53.815 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:38:54.641 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[38286]: Deploying daemon mds.foo.smithi178.jvpymc on smithi178 2024-06-14T22:38:54.641 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[38286]: mds.? [v2:172.21.15.46:6834/1593007743,v1:172.21.15.46:6835/1593007743] up:boot 2024-06-14T22:38:54.642 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[38286]: daemon mds.foo.smithi046.fdgzyw assigned to filesystem foo as rank 0 (now has 1 ranks) 2024-06-14T22:38:54.642 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[38286]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-14T22:38:54.642 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[38286]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-14T22:38:54.642 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[38286]: Cluster is now healthy 2024-06-14T22:38:54.642 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[38286]: fsmap foo:0 1 up:standby 2024-06-14T22:38:54.642 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "foo.smithi046.fdgzyw"}]: dispatch 2024-06-14T22:38:54.643 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[38286]: fsmap foo:1 {0=foo.smithi046.fdgzyw=up:creating} 2024-06-14T22:38:54.643 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[38286]: daemon mds.foo.smithi046.fdgzyw is now active in filesystem foo as rank 0 2024-06-14T22:38:54.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[29586]: Deploying daemon mds.foo.smithi178.jvpymc on smithi178 2024-06-14T22:38:54.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[29586]: mds.? [v2:172.21.15.46:6834/1593007743,v1:172.21.15.46:6835/1593007743] up:boot 2024-06-14T22:38:54.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[29586]: daemon mds.foo.smithi046.fdgzyw assigned to filesystem foo as rank 0 (now has 1 ranks) 2024-06-14T22:38:54.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[29586]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-14T22:38:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[29586]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-14T22:38:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[29586]: Cluster is now healthy 2024-06-14T22:38:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[29586]: fsmap foo:0 1 up:standby 2024-06-14T22:38:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "foo.smithi046.fdgzyw"}]: dispatch 2024-06-14T22:38:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[29586]: fsmap foo:1 {0=foo.smithi046.fdgzyw=up:creating} 2024-06-14T22:38:54.644 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:54 smithi046 ceph-mon[29586]: daemon mds.foo.smithi046.fdgzyw is now active in filesystem foo as rank 0 2024-06-14T22:38:54.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:54 smithi178 ceph-mon[32942]: Deploying daemon mds.foo.smithi178.jvpymc on smithi178 2024-06-14T22:38:54.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:54 smithi178 ceph-mon[32942]: mds.? [v2:172.21.15.46:6834/1593007743,v1:172.21.15.46:6835/1593007743] up:boot 2024-06-14T22:38:54.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:54 smithi178 ceph-mon[32942]: daemon mds.foo.smithi046.fdgzyw assigned to filesystem foo as rank 0 (now has 1 ranks) 2024-06-14T22:38:54.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:54 smithi178 ceph-mon[32942]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-14T22:38:54.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:54 smithi178 ceph-mon[32942]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-14T22:38:54.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:54 smithi178 ceph-mon[32942]: Cluster is now healthy 2024-06-14T22:38:54.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:54 smithi178 ceph-mon[32942]: fsmap foo:0 1 up:standby 2024-06-14T22:38:54.741 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:54 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "foo.smithi046.fdgzyw"}]: dispatch 2024-06-14T22:38:54.742 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:54 smithi178 ceph-mon[32942]: fsmap foo:1 {0=foo.smithi046.fdgzyw=up:creating} 2024-06-14T22:38:54.742 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:54 smithi178 ceph-mon[32942]: daemon mds.foo.smithi046.fdgzyw is now active in filesystem foo as rank 0 2024-06-14T22:38:55.544 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:55 smithi046 ceph-mon[38286]: pgmap v53: 73 pgs: 73 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:55.558 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:55 smithi046 ceph-mon[38286]: mds.? [v2:172.21.15.46:6834/1593007743,v1:172.21.15.46:6835/1593007743] up:active 2024-06-14T22:38:55.558 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:55 smithi046 ceph-mon[38286]: fsmap foo:1 {0=foo.smithi046.fdgzyw=up:active} 2024-06-14T22:38:55.558 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:55 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:55.558 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:55 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:55.558 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:55 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:55.558 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:55 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:55.559 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:55 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:55.559 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:55 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:38:55.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:55 smithi046 ceph-mon[29586]: pgmap v53: 73 pgs: 73 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:55.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:55 smithi046 ceph-mon[29586]: mds.? [v2:172.21.15.46:6834/1593007743,v1:172.21.15.46:6835/1593007743] up:active 2024-06-14T22:38:55.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:55 smithi046 ceph-mon[29586]: fsmap foo:1 {0=foo.smithi046.fdgzyw=up:active} 2024-06-14T22:38:55.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:55 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:55.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:55 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:55.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:55 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:55.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:55 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:55.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:55 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:55.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:55 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:38:55.696 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:38:55.696 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":69,"fsid":"cb009720-2a9d-11ef-bc9d-c7b262605968","created":"2024-06-14T22:31:50.709367+0000","modified":"2024-06-14T22:38:44.485850+0000","last_up_change":"2024-06-14T22:36:57.461962+0000","last_in_change":"2024-06-14T22:36:40.661606+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-14T22:34:58.841776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-06-14T22:37:30.106154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs.foo.meta","create_time":"2024-06-14T22:38:41.414985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"foo"}}},{"pool":4,"pool_name":"cephfs.foo.data","create_time":"2024-06-14T22:38:41.923807+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"foo"}}}],"osds":[{"osd":0,"uuid":"33945598-f864-40e6-904e-1bff72ec8b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6803","nonce":2467893990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6805","nonce":2467893990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6809","nonce":2467893990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6807","nonce":2467893990}]},"public_addr":"172.21.15.46:6803/2467893990","cluster_addr":"172.21.15.46:6805/2467893990","heartbeat_back_addr":"172.21.15.46:6809/2467893990","heartbeat_front_addr":"172.21.15.46:6807/2467893990","state":["exists","up"]},{"osd":1,"uuid":"48f0e5ef-2d0a-4858-a629-69f82c604f9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6811","nonce":2133775041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6813","nonce":2133775041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6817","nonce":2133775041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6815","nonce":2133775041}]},"public_addr":"172.21.15.46:6811/2133775041","cluster_addr":"172.21.15.46:6813/2133775041","heartbeat_back_addr":"172.21.15.46:6817/2133775041","heartbeat_front_addr":"172.21.15.46:6815/2133775041","state":["exists","up"]},{"osd":2,"uuid":"74b998f5-fb99-4c37-9463-11dd602f1364","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6819","nonce":3036855066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6821","nonce":3036855066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6825","nonce":3036855066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6823","nonce":3036855066}]},"public_addr":"172.21.15.46:6819/3036855066","cluster_addr":"172.21.15.46:6821/3036855066","heartbeat_back_addr":"172.21.15.46:6825/3036855066","heartbeat_front_addr":"172.21.15.46:6823/3036855066","state":["exists","up"]},{"osd":3,"uuid":"e4bd323c-2885-47a0-a44c-275077fc4b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6827","nonce":2597052465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6829","nonce":2597052465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6833","nonce":2597052465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6831","nonce":2597052465}]},"public_addr":"172.21.15.46:6827/2597052465","cluster_addr":"172.21.15.46:6829/2597052465","heartbeat_back_addr":"172.21.15.46:6833/2597052465","heartbeat_front_addr":"172.21.15.46:6831/2597052465","state":["exists","up"]},{"osd":4,"uuid":"0a0b2d3d-3e61-47be-a95f-8294f79ecc7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6801","nonce":2179289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6803","nonce":2179289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6807","nonce":2179289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6805","nonce":2179289524}]},"public_addr":"172.21.15.178:6801/2179289524","cluster_addr":"172.21.15.178:6803/2179289524","heartbeat_back_addr":"172.21.15.178:6807/2179289524","heartbeat_front_addr":"172.21.15.178:6805/2179289524","state":["exists","up"]},{"osd":5,"uuid":"29f79df4-039f-44a3-8570-7252eca3abe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6809","nonce":2813027226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6811","nonce":2813027226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6815","nonce":2813027226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6813","nonce":2813027226}]},"public_addr":"172.21.15.178:6809/2813027226","cluster_addr":"172.21.15.178:6811/2813027226","heartbeat_back_addr":"172.21.15.178:6815/2813027226","heartbeat_front_addr":"172.21.15.178:6813/2813027226","state":["exists","up"]},{"osd":6,"uuid":"56661a2b-4e13-47e7-972a-bb1861d5e33d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6817","nonce":3396977870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6819","nonce":3396977870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6823","nonce":3396977870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6821","nonce":3396977870}]},"public_addr":"172.21.15.178:6817/3396977870","cluster_addr":"172.21.15.178:6819/3396977870","heartbeat_back_addr":"172.21.15.178:6823/3396977870","heartbeat_front_addr":"172.21.15.178:6821/3396977870","state":["exists","up"]},{"osd":7,"uuid":"205c8b05-359d-4ff3-a158-3e80e1c5fda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6825","nonce":826597267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6827","nonce":826597267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6831","nonce":826597267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6829","nonce":826597267}]},"public_addr":"172.21.15.178:6825/826597267","cluster_addr":"172.21.15.178:6827/826597267","heartbeat_back_addr":"172.21.15.178:6831/826597267","heartbeat_front_addr":"172.21.15.178:6829/826597267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:34:06.575453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:34:31.020820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:34:55.617857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:35:20.862506+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:35:42.941353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:36:06.653807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:36:31.539489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:36:56.167994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/1358765809":"2024-06-15T22:37:30.185199+0000","172.21.15.46:0/1856025406":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/2833589127":"2024-06-15T22:32:44.342121+0000","172.21.15.46:6800/1612763800":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/662686570":"2024-06-15T22:37:30.185199+0000","172.21.15.46:6801/1612763800":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/4205563548":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/1630559323":"2024-06-15T22:32:44.342121+0000","172.21.15.46:6801/873354145":"2024-06-15T22:37:30.185199+0000","172.21.15.46:0/3550067878":"2024-06-15T22:32:44.342121+0000","172.21.15.46:0/2045722581":"2024-06-15T22:37:30.185199+0000","172.21.15.46:0/96950257":"2024-06-15T22:32:22.225646+0000","172.21.15.46:6800/4238739813":"2024-06-15T22:32:44.342121+0000","172.21.15.46:6801/4238739813":"2024-06-15T22:32:44.342121+0000","172.21.15.46:0/2079110403":"2024-06-15T22:37:30.185199+0000","172.21.15.46:6800/873354145":"2024-06-15T22:37:30.185199+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-14T22:38:55.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:55 smithi178 ceph-mon[32942]: pgmap v53: 73 pgs: 73 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:38:55.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:55 smithi178 ceph-mon[32942]: mds.? [v2:172.21.15.46:6834/1593007743,v1:172.21.15.46:6835/1593007743] up:active 2024-06-14T22:38:55.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:55 smithi178 ceph-mon[32942]: fsmap foo:1 {0=foo.smithi046.fdgzyw=up:active} 2024-06-14T22:38:55.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:55 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:55.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:55 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:55.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:55 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:55.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:55 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:55.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:55 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:55.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:55 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:38:56.378 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-14T22:38:56.379 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd dump --format=json 2024-06-14T22:38:56.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:56 smithi046 ceph-mon[38286]: mds.? [v2:172.21.15.178:6832/668207665,v1:172.21.15.178:6833/668207665] up:boot 2024-06-14T22:38:56.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:56 smithi046 ceph-mon[38286]: fsmap foo:1 {0=foo.smithi046.fdgzyw=up:active} 1 up:standby 2024-06-14T22:38:56.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:56 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "foo.smithi178.jvpymc"}]: dispatch 2024-06-14T22:38:56.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:56 smithi046 ceph-mon[38286]: fsmap foo:1 {0=foo.smithi046.fdgzyw=up:active} 1 up:standby 2024-06-14T22:38:56.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:56 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/1107085417' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-14T22:38:56.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:56 smithi046 ceph-mon[29586]: mds.? [v2:172.21.15.178:6832/668207665,v1:172.21.15.178:6833/668207665] up:boot 2024-06-14T22:38:56.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:56 smithi046 ceph-mon[29586]: fsmap foo:1 {0=foo.smithi046.fdgzyw=up:active} 1 up:standby 2024-06-14T22:38:56.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:56 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "foo.smithi178.jvpymc"}]: dispatch 2024-06-14T22:38:56.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:56 smithi046 ceph-mon[29586]: fsmap foo:1 {0=foo.smithi046.fdgzyw=up:active} 1 up:standby 2024-06-14T22:38:56.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:56 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/1107085417' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-14T22:38:56.734 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:56 smithi178 ceph-mon[32942]: mds.? [v2:172.21.15.178:6832/668207665,v1:172.21.15.178:6833/668207665] up:boot 2024-06-14T22:38:56.734 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:56 smithi178 ceph-mon[32942]: fsmap foo:1 {0=foo.smithi046.fdgzyw=up:active} 1 up:standby 2024-06-14T22:38:56.734 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:56 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "mds metadata", "who": "foo.smithi178.jvpymc"}]: dispatch 2024-06-14T22:38:56.734 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:56 smithi178 ceph-mon[32942]: fsmap foo:1 {0=foo.smithi046.fdgzyw=up:active} 1 up:standby 2024-06-14T22:38:56.734 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:56 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/1107085417' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-14T22:38:57.142 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:38:57.531 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:57 smithi046 ceph-mon[29586]: pgmap v54: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-14T22:38:57.531 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:57 smithi046 ceph-mon[38286]: pgmap v54: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-14T22:38:57.712 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:38:57 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:38:57] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-06-14T22:38:57.713 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:57 smithi178 ceph-mon[32942]: pgmap v54: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-14T22:38:58.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:58 smithi046 ceph-mon[29586]: mds.? [v2:172.21.15.46:6834/1593007743,v1:172.21.15.46:6835/1593007743] up:active 2024-06-14T22:38:58.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:58 smithi046 ceph-mon[29586]: fsmap foo:1 {0=foo.smithi046.fdgzyw=up:active} 1 up:standby 2024-06-14T22:38:58.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:58 smithi046 ceph-mon[38286]: mds.? [v2:172.21.15.46:6834/1593007743,v1:172.21.15.46:6835/1593007743] up:active 2024-06-14T22:38:58.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:58 smithi046 ceph-mon[38286]: fsmap foo:1 {0=foo.smithi046.fdgzyw=up:active} 1 up:standby 2024-06-14T22:38:58.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:58 smithi178 ceph-mon[32942]: mds.? [v2:172.21.15.46:6834/1593007743,v1:172.21.15.46:6835/1593007743] up:active 2024-06-14T22:38:58.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:58 smithi178 ceph-mon[32942]: fsmap foo:1 {0=foo.smithi046.fdgzyw=up:active} 1 up:standby 2024-06-14T22:38:58.879 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:38:58.879 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":69,"fsid":"cb009720-2a9d-11ef-bc9d-c7b262605968","created":"2024-06-14T22:31:50.709367+0000","modified":"2024-06-14T22:38:44.485850+0000","last_up_change":"2024-06-14T22:36:57.461962+0000","last_in_change":"2024-06-14T22:36:40.661606+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-14T22:34:58.841776+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-06-14T22:37:30.106154+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs.foo.meta","create_time":"2024-06-14T22:38:41.414985+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"foo"}}},{"pool":4,"pool_name":"cephfs.foo.data","create_time":"2024-06-14T22:38:41.923807+0000","flags":131073,"flags_names":"hashpspool,bulk","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"foo"}}}],"osds":[{"osd":0,"uuid":"33945598-f864-40e6-904e-1bff72ec8b04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6803","nonce":2467893990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6805","nonce":2467893990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6809","nonce":2467893990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":2467893990},{"type":"v1","addr":"172.21.15.46:6807","nonce":2467893990}]},"public_addr":"172.21.15.46:6803/2467893990","cluster_addr":"172.21.15.46:6805/2467893990","heartbeat_back_addr":"172.21.15.46:6809/2467893990","heartbeat_front_addr":"172.21.15.46:6807/2467893990","state":["exists","up"]},{"osd":1,"uuid":"48f0e5ef-2d0a-4858-a629-69f82c604f9b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6811","nonce":2133775041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6813","nonce":2133775041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6817","nonce":2133775041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":2133775041},{"type":"v1","addr":"172.21.15.46:6815","nonce":2133775041}]},"public_addr":"172.21.15.46:6811/2133775041","cluster_addr":"172.21.15.46:6813/2133775041","heartbeat_back_addr":"172.21.15.46:6817/2133775041","heartbeat_front_addr":"172.21.15.46:6815/2133775041","state":["exists","up"]},{"osd":2,"uuid":"74b998f5-fb99-4c37-9463-11dd602f1364","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6819","nonce":3036855066}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6821","nonce":3036855066}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6825","nonce":3036855066}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":3036855066},{"type":"v1","addr":"172.21.15.46:6823","nonce":3036855066}]},"public_addr":"172.21.15.46:6819/3036855066","cluster_addr":"172.21.15.46:6821/3036855066","heartbeat_back_addr":"172.21.15.46:6825/3036855066","heartbeat_front_addr":"172.21.15.46:6823/3036855066","state":["exists","up"]},{"osd":3,"uuid":"e4bd323c-2885-47a0-a44c-275077fc4b06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6827","nonce":2597052465}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6829","nonce":2597052465}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6833","nonce":2597052465}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":2597052465},{"type":"v1","addr":"172.21.15.46:6831","nonce":2597052465}]},"public_addr":"172.21.15.46:6827/2597052465","cluster_addr":"172.21.15.46:6829/2597052465","heartbeat_back_addr":"172.21.15.46:6833/2597052465","heartbeat_front_addr":"172.21.15.46:6831/2597052465","state":["exists","up"]},{"osd":4,"uuid":"0a0b2d3d-3e61-47be-a95f-8294f79ecc7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6801","nonce":2179289524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6803","nonce":2179289524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6807","nonce":2179289524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":2179289524},{"type":"v1","addr":"172.21.15.178:6805","nonce":2179289524}]},"public_addr":"172.21.15.178:6801/2179289524","cluster_addr":"172.21.15.178:6803/2179289524","heartbeat_back_addr":"172.21.15.178:6807/2179289524","heartbeat_front_addr":"172.21.15.178:6805/2179289524","state":["exists","up"]},{"osd":5,"uuid":"29f79df4-039f-44a3-8570-7252eca3abe8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6809","nonce":2813027226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6811","nonce":2813027226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6815","nonce":2813027226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":2813027226},{"type":"v1","addr":"172.21.15.178:6813","nonce":2813027226}]},"public_addr":"172.21.15.178:6809/2813027226","cluster_addr":"172.21.15.178:6811/2813027226","heartbeat_back_addr":"172.21.15.178:6815/2813027226","heartbeat_front_addr":"172.21.15.178:6813/2813027226","state":["exists","up"]},{"osd":6,"uuid":"56661a2b-4e13-47e7-972a-bb1861d5e33d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6817","nonce":3396977870}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6819","nonce":3396977870}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6823","nonce":3396977870}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3396977870},{"type":"v1","addr":"172.21.15.178:6821","nonce":3396977870}]},"public_addr":"172.21.15.178:6817/3396977870","cluster_addr":"172.21.15.178:6819/3396977870","heartbeat_back_addr":"172.21.15.178:6823/3396977870","heartbeat_front_addr":"172.21.15.178:6821/3396977870","state":["exists","up"]},{"osd":7,"uuid":"205c8b05-359d-4ff3-a158-3e80e1c5fda6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6825","nonce":826597267}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6827","nonce":826597267}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6830","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6831","nonce":826597267}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6828","nonce":826597267},{"type":"v1","addr":"172.21.15.178:6829","nonce":826597267}]},"public_addr":"172.21.15.178:6825/826597267","cluster_addr":"172.21.15.178:6827/826597267","heartbeat_back_addr":"172.21.15.178:6831/826597267","heartbeat_front_addr":"172.21.15.178:6829/826597267","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:34:06.575453+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:34:31.020820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:34:55.617857+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:35:20.862506+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:35:42.941353+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:36:06.653807+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:36:31.539489+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-06-14T22:36:56.167994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/1358765809":"2024-06-15T22:37:30.185199+0000","172.21.15.46:0/1856025406":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/2833589127":"2024-06-15T22:32:44.342121+0000","172.21.15.46:6800/1612763800":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/662686570":"2024-06-15T22:37:30.185199+0000","172.21.15.46:6801/1612763800":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/4205563548":"2024-06-15T22:32:22.225646+0000","172.21.15.46:0/1630559323":"2024-06-15T22:32:44.342121+0000","172.21.15.46:6801/873354145":"2024-06-15T22:37:30.185199+0000","172.21.15.46:0/3550067878":"2024-06-15T22:32:44.342121+0000","172.21.15.46:0/2045722581":"2024-06-15T22:37:30.185199+0000","172.21.15.46:0/96950257":"2024-06-15T22:32:22.225646+0000","172.21.15.46:6800/4238739813":"2024-06-15T22:32:44.342121+0000","172.21.15.46:6801/4238739813":"2024-06-15T22:32:44.342121+0000","172.21.15.46:0/2079110403":"2024-06-15T22:37:30.185199+0000","172.21.15.46:6800/873354145":"2024-06-15T22:37:30.185199+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-14T22:38:59.569 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-14T22:38:59.570 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-14T22:38:59.570 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-14T22:38:59.571 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-14T22:38:59.571 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-14T22:38:59.572 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-14T22:38:59.572 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-14T22:38:59.573 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-14T22:38:59.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:59 smithi178 ceph-mon[32942]: pgmap v55: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-14T22:38:59.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:59 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/1358481371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-14T22:38:59.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:59 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:59.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:38:59 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:59.943 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:59 smithi046 ceph-mon[29586]: pgmap v55: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-14T22:38:59.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:59 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/1358481371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-14T22:38:59.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:59 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:59.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:38:59 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:59.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:59 smithi046 ceph-mon[38286]: pgmap v55: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-14T22:38:59.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:59 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/1358481371' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-14T22:38:59.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:59 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:38:59.944 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:38:59 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:01.199 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:01.205 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:01.205 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:01.205 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:01.220 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:00 smithi046 ceph-mon[38286]: mds.? [v2:172.21.15.178:6832/668207665,v1:172.21.15.178:6833/668207665] up:standby 2024-06-14T22:39:01.220 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:00 smithi046 ceph-mon[38286]: fsmap foo:1 {0=foo.smithi046.fdgzyw=up:active} 1 up:standby 2024-06-14T22:39:01.220 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:00 smithi046 ceph-mon[38286]: pgmap v56: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-14T22:39:01.220 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:00 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:01.220 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:00 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:39:01.221 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:00 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:01.221 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:00 smithi046 ceph-mon[29586]: mds.? [v2:172.21.15.178:6832/668207665,v1:172.21.15.178:6833/668207665] up:standby 2024-06-14T22:39:01.221 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:00 smithi046 ceph-mon[29586]: fsmap foo:1 {0=foo.smithi046.fdgzyw=up:active} 1 up:standby 2024-06-14T22:39:01.222 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:00 smithi046 ceph-mon[29586]: pgmap v56: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-14T22:39:01.222 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:00 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:01.222 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:00 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:39:01.222 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:00 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:01.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:00 smithi178 ceph-mon[32942]: mds.? [v2:172.21.15.178:6832/668207665,v1:172.21.15.178:6833/668207665] up:standby 2024-06-14T22:39:01.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:00 smithi178 ceph-mon[32942]: fsmap foo:1 {0=foo.smithi046.fdgzyw=up:active} 1 up:standby 2024-06-14T22:39:01.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:00 smithi178 ceph-mon[32942]: pgmap v56: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-14T22:39:01.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:00 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:01.275 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:00 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:39:01.276 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:00 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:01.573 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:01 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:39:01] "GET /metrics HTTP/1.1" 200 239135 "" "Prometheus/2.43.0" 2024-06-14T22:39:01.731 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:01.731 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:02.254 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:03.341 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:03 smithi046 ceph-mon[38286]: pgmap v57: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-14T22:39:03.341 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:03 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:03.341 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:03 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:03.342 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:03 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:39:03.342 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:03 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:39:03.342 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:03 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:03.342 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:03 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:39:03.342 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:03 smithi046 ceph-mon[29586]: pgmap v57: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-14T22:39:03.342 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:03 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:03.342 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:03 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:03.342 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:03 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:39:03.343 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:03 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:39:03.343 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:03 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:03.343 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:03 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:39:03.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:03 smithi178 ceph-mon[32942]: pgmap v57: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-14T22:39:03.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:03 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:03.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:03 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:03.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:03 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:39:03.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:03 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:39:03.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:03 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:03.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:03 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-14T22:39:05.210 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:05.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:05 smithi178 ceph-mon[32942]: pgmap v58: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-14T22:39:05.675 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:05 smithi046 ceph-mon[29586]: pgmap v58: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-14T22:39:05.676 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:05 smithi046 ceph-mon[38286]: pgmap v58: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-14T22:39:06.104 INFO:teuthology.orchestra.run.smithi046.stdout:77309411380 2024-06-14T22:39:06.105 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-14T22:39:07.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:07 smithi178 ceph-mon[32942]: pgmap v59: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-14T22:39:07.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:07 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:07.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:07 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:07.622 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:07 smithi046 ceph-mon[29586]: pgmap v59: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-14T22:39:07.622 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:07 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:07.622 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:07 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:07.622 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:07 smithi046 ceph-mon[38286]: pgmap v59: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s wr, 3 op/s 2024-06-14T22:39:07.622 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:07 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:07.622 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:07 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:08.025 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:39:07 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:39:07] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-06-14T22:39:09.314 INFO:teuthology.orchestra.run.smithi046.stdout:111669149744 2024-06-14T22:39:09.315 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-14T22:39:09.523 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:09 smithi046 ceph-mon[38286]: pgmap v60: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-06-14T22:39:09.523 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:09 smithi046 ceph-mon[29586]: pgmap v60: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-06-14T22:39:09.525 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:09 smithi178 ceph-mon[32942]: pgmap v60: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-06-14T22:39:11.464 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:11 smithi046 ceph-mon[29586]: pgmap v61: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-06-14T22:39:11.464 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:11 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:39:11] "GET /metrics HTTP/1.1" 200 239135 "" "Prometheus/2.43.0" 2024-06-14T22:39:11.464 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:11 smithi046 ceph-mon[38286]: pgmap v61: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-06-14T22:39:11.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:11 smithi178 ceph-mon[32942]: pgmap v61: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-06-14T22:39:12.702 INFO:teuthology.orchestra.run.smithi046.stdout:201863462947 2024-06-14T22:39:12.702 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-14T22:39:13.678 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:13 smithi046 ceph-mon[29586]: pgmap v62: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-06-14T22:39:13.679 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:13 smithi046 ceph-mon[38286]: pgmap v62: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-06-14T22:39:13.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:13 smithi178 ceph-mon[32942]: pgmap v62: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s wr, 0 op/s 2024-06-14T22:39:15.627 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:15 smithi046 ceph-mon[29586]: pgmap v63: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:15.627 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:15 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:39:15.627 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:15 smithi046 ceph-mon[38286]: pgmap v63: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:15.627 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:15 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:39:15.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:15 smithi178 ceph-mon[32942]: pgmap v63: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:15.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:15 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:39:15.910 INFO:teuthology.orchestra.run.smithi046.stdout:167503724584 2024-06-14T22:39:15.910 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-14T22:39:16.650 INFO:teuthology.orchestra.run.smithi046.stdout:34359738432 2024-06-14T22:39:16.650 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-14T22:39:17.018 INFO:teuthology.orchestra.run.smithi046.stdout:55834574907 2024-06-14T22:39:17.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-14T22:39:17.445 INFO:teuthology.orchestra.run.smithi046.stdout:133143986220 2024-06-14T22:39:17.445 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-14T22:39:17.552 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:17 smithi178 ceph-mon[32942]: pgmap v64: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:17.655 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:17 smithi046 ceph-mon[29586]: pgmap v64: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:17.656 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:17 smithi046 ceph-mon[38286]: pgmap v64: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:18.025 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:39:17 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:39:17] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-06-14T22:39:18.461 INFO:teuthology.orchestra.run.smithi046.stdout:236223201311 2024-06-14T22:39:18.462 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-14T22:39:18.493 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:18.562 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:18.798 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:18.799 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:19.311 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:19 smithi046 ceph-mon[29586]: pgmap v65: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:19.311 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:19 smithi046 ceph-mon[38286]: pgmap v65: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:19.684 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:19.685 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:19.685 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:19.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:19 smithi178 ceph-mon[32942]: pgmap v65: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:21.348 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:21 smithi046 ceph-mon[29586]: pgmap v66: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:21.348 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:21 smithi046 ceph-mon[38286]: pgmap v66: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:21.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:21 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:39:21] "GET /metrics HTTP/1.1" 200 239107 "" "Prometheus/2.43.0" 2024-06-14T22:39:21.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:21 smithi178 ceph-mon[32942]: pgmap v66: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:23.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:23 smithi046 ceph-mon[29586]: pgmap v67: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:23.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:23 smithi046 ceph-mon[38286]: pgmap v67: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:23.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:23 smithi178 ceph-mon[32942]: pgmap v67: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:24.500 INFO:teuthology.orchestra.run.smithi046.stdout:34359738434 2024-06-14T22:39:25.320 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:25 smithi046 ceph-mon[29586]: pgmap v68: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:25.320 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:25 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/1448004899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-14T22:39:25.321 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:25 smithi046 ceph-mon[38286]: pgmap v68: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:25.321 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:25 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/1448004899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-14T22:39:25.564 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:25.577 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:25 smithi178 ceph-mon[32942]: pgmap v68: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:25.578 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:25 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/1448004899' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-14T22:39:26.420 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738432 got 34359738434 for osd.0 2024-06-14T22:39:26.420 DEBUG:teuthology.parallel:result is None 2024-06-14T22:39:26.597 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:26 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/2034241324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-14T22:39:26.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:26 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/2034241324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-14T22:39:26.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:26 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/2034241324' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-14T22:39:27.411 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:27 smithi046 ceph-mon[29586]: pgmap v69: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:27.411 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:27 smithi046 ceph-mon[38286]: pgmap v69: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:27.552 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:27 smithi178 ceph-mon[32942]: pgmap v69: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:28.025 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:39:27 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:39:27] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-06-14T22:39:28.759 INFO:teuthology.orchestra.run.smithi046.stdout:55834574909 2024-06-14T22:39:29.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:29 smithi046 ceph-mon[29586]: pgmap v70: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:29.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:29 smithi046 ceph-mon[38286]: pgmap v70: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:29.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:29 smithi178 ceph-mon[32942]: pgmap v70: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:30.271 INFO:teuthology.orchestra.run.smithi046.stdout:77309411385 2024-06-14T22:39:30.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:30 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-06-14T22:39:30.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:30 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-06-14T22:39:30.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:30 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-06-14T22:39:30.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:30 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:39:30.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:30 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/3578774216' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-14T22:39:30.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:30 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-06-14T22:39:30.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:30 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-06-14T22:39:30.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:30 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-06-14T22:39:30.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:30 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:39:30.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:30 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/3578774216' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-14T22:39:30.589 INFO:teuthology.orchestra.run.smithi046.stdout:167503724587 2024-06-14T22:39:30.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:30 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-06-14T22:39:30.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:30 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-06-14T22:39:30.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:30 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]: dispatch 2024-06-14T22:39:30.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:30 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:39:30.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:30 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/3578774216' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-14T22:39:31.228 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574907 got 55834574909 for osd.1 2024-06-14T22:39:31.228 DEBUG:teuthology.parallel:result is None 2024-06-14T22:39:31.455 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:31 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:39:31] "GET /metrics HTTP/1.1" 200 239073 "" "Prometheus/2.43.0" 2024-06-14T22:39:31.455 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:31 smithi046 ceph-mon[38286]: pgmap v71: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:31.455 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-06-14T22:39:31.455 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-06-14T22:39:31.455 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:31 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-06-14T22:39:31.455 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:31 smithi046 ceph-mon[38286]: osdmap e70: 8 total, 8 up, 8 in 2024-06-14T22:39:31.455 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:31 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/15674168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-14T22:39:31.456 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:31 smithi046 ceph-mon[29586]: pgmap v71: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:31.456 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-06-14T22:39:31.456 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-06-14T22:39:31.456 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:31 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-06-14T22:39:31.456 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:31 smithi046 ceph-mon[29586]: osdmap e70: 8 total, 8 up, 8 in 2024-06-14T22:39:31.456 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:31 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/15674168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-14T22:39:31.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:31 smithi178 ceph-mon[32942]: pgmap v71: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:31.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-06-14T22:39:31.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-06-14T22:39:31.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:31 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1d", "id": [1, 2]}]': finished 2024-06-14T22:39:31.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:31 smithi178 ceph-mon[32942]: osdmap e70: 8 total, 8 up, 8 in 2024-06-14T22:39:31.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:31 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/15674168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-14T22:39:31.856 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724584 got 167503724587 for osd.5 2024-06-14T22:39:31.856 DEBUG:teuthology.parallel:result is None 2024-06-14T22:39:32.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:32 smithi046 ceph-mon[29586]: osdmap e71: 8 total, 8 up, 8 in 2024-06-14T22:39:32.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:32 smithi046 ceph-mon[38286]: osdmap e71: 8 total, 8 up, 8 in 2024-06-14T22:39:32.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:32 smithi178 ceph-mon[32942]: osdmap e71: 8 total, 8 up, 8 in 2024-06-14T22:39:33.251 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411380 got 77309411385 for osd.2 2024-06-14T22:39:33.251 DEBUG:teuthology.parallel:result is None 2024-06-14T22:39:33.518 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:33 smithi046 ceph-mon[29586]: pgmap v74: 73 pgs: 3 peering, 70 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:33.518 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:33 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/2440041188' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-14T22:39:33.518 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:33 smithi046 ceph-mon[38286]: pgmap v74: 73 pgs: 3 peering, 70 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:33.518 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:33 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/2440041188' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-14T22:39:33.621 INFO:teuthology.orchestra.run.smithi046.stdout:201863462952 2024-06-14T22:39:33.700 INFO:teuthology.orchestra.run.smithi046.stdout:111669149749 2024-06-14T22:39:33.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:33 smithi178 ceph-mon[32942]: pgmap v74: 73 pgs: 3 peering, 70 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:33.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:33 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/2440041188' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-14T22:39:34.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:34 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/2899168030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-14T22:39:34.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:34 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/2712659481' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-14T22:39:34.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:34 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/2899168030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-14T22:39:34.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:34 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/2712659481' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-14T22:39:34.569 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149744 got 111669149749 for osd.3 2024-06-14T22:39:34.569 DEBUG:teuthology.parallel:result is None 2024-06-14T22:39:34.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:34 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/2899168030' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-14T22:39:34.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:34 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/2712659481' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-14T22:39:35.101 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462947 got 201863462952 for osd.6 2024-06-14T22:39:35.101 DEBUG:teuthology.parallel:result is None 2024-06-14T22:39:35.200 INFO:teuthology.orchestra.run.smithi046.stdout:133143986224 2024-06-14T22:39:35.471 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:35 smithi046 ceph-mon[29586]: pgmap v75: 73 pgs: 3 peering, 70 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:35.471 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:35 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/950792926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-14T22:39:35.471 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:35 smithi046 ceph-mon[38286]: pgmap v75: 73 pgs: 3 peering, 70 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:35.471 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:35 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/950792926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-14T22:39:35.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:35 smithi178 ceph-mon[32942]: pgmap v75: 73 pgs: 3 peering, 70 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:35.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:35 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/950792926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-14T22:39:35.827 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986220 got 133143986224 for osd.4 2024-06-14T22:39:35.827 DEBUG:teuthology.parallel:result is None 2024-06-14T22:39:36.005 INFO:teuthology.orchestra.run.smithi046.stdout:236223201314 2024-06-14T22:39:36.859 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201311 got 236223201314 for osd.7 2024-06-14T22:39:36.859 DEBUG:teuthology.parallel:result is None 2024-06-14T22:39:36.859 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-14T22:39:36.859 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph pg dump --format=json 2024-06-14T22:39:37.326 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:37.552 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:37 smithi178 ceph-mon[32942]: pgmap v76: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:37.624 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:37 smithi046 ceph-mon[29586]: pgmap v76: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:37.624 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:37 smithi046 ceph-mon[38286]: pgmap v76: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:37.856 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:39:37 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:39:37] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-06-14T22:39:39.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:39 smithi046 ceph-mon[29586]: pgmap v77: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:39.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:39 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:39.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:39 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:39.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:39 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:39:39.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:39 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:39:39.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:39 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:39.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:39 smithi046 ceph-mon[38286]: pgmap v77: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:39.443 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:39 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:39.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:39 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:39.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:39 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:39:39.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:39 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:39:39.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:39 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:39.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:39 smithi178 ceph-mon[32942]: pgmap v77: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:39.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:39 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:39.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:39 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:39.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:39 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-14T22:39:39.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:39 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-14T22:39:39.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:39 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' 2024-06-14T22:39:39.869 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:39:39.871 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-06-14T22:39:40.535 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2024-06-14T22:39:38.223017+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":219,"acting":219,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":222,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2386332,"kb_used_data":7468,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747574884,"statfs":{"total":767960285184,"available":765516681216,"internally_reserved":0,"allocated":7647232,"data_stored":4308306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001488"},"pg_stats":[{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316868+0000","last_change":"2024-06-14T22:38:43.725693+0000","last_active":"2024-06-14T22:39:30.316868+0000","last_peered":"2024-06-14T22:39:30.316868+0000","last_clean":"2024-06-14T22:39:30.316868+0000","last_became_active":"2024-06-14T22:38:43.725411+0000","last_became_peered":"2024-06-14T22:38:43.725411+0000","last_unstale":"2024-06-14T22:39:30.316868+0000","last_undegraded":"2024-06-14T22:39:30.316868+0000","last_fullsized":"2024-06-14T22:39:30.316868+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T00:22:31.533026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316562+0000","last_change":"2024-06-14T22:38:42.485033+0000","last_active":"2024-06-14T22:39:30.316562+0000","last_peered":"2024-06-14T22:39:30.316562+0000","last_clean":"2024-06-14T22:39:30.316562+0000","last_became_active":"2024-06-14T22:38:42.484143+0000","last_became_peered":"2024-06-14T22:38:42.484143+0000","last_unstale":"2024-06-14T22:39:30.316562+0000","last_undegraded":"2024-06-14T22:39:30.316562+0000","last_fullsized":"2024-06-14T22:39:30.316562+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:44:59.321972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312519+0000","last_change":"2024-06-14T22:38:43.226317+0000","last_active":"2024-06-14T22:39:30.312519+0000","last_peered":"2024-06-14T22:39:30.312519+0000","last_clean":"2024-06-14T22:39:30.312519+0000","last_became_active":"2024-06-14T22:38:43.226041+0000","last_became_peered":"2024-06-14T22:38:43.226041+0000","last_unstale":"2024-06-14T22:39:30.312519+0000","last_undegraded":"2024-06-14T22:39:30.312519+0000","last_fullsized":"2024-06-14T22:39:30.312519+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T09:01:07.074548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312628+0000","last_change":"2024-06-14T22:38:42.489752+0000","last_active":"2024-06-14T22:39:30.312628+0000","last_peered":"2024-06-14T22:39:30.312628+0000","last_clean":"2024-06-14T22:39:30.312628+0000","last_became_active":"2024-06-14T22:38:42.489474+0000","last_became_peered":"2024-06-14T22:38:42.489474+0000","last_unstale":"2024-06-14T22:39:30.312628+0000","last_undegraded":"2024-06-14T22:39:30.312628+0000","last_fullsized":"2024-06-14T22:39:30.312628+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T03:03:35.515139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313406+0000","last_change":"2024-06-14T22:38:43.726372+0000","last_active":"2024-06-14T22:39:30.313406+0000","last_peered":"2024-06-14T22:39:30.313406+0000","last_clean":"2024-06-14T22:39:30.313406+0000","last_became_active":"2024-06-14T22:38:43.726078+0000","last_became_peered":"2024-06-14T22:38:43.726078+0000","last_unstale":"2024-06-14T22:39:30.313406+0000","last_undegraded":"2024-06-14T22:39:30.313406+0000","last_fullsized":"2024-06-14T22:39:30.313406+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:05:21.767046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317236+0000","last_change":"2024-06-14T22:38:42.485551+0000","last_active":"2024-06-14T22:39:30.317236+0000","last_peered":"2024-06-14T22:39:30.317236+0000","last_clean":"2024-06-14T22:39:30.317236+0000","last_became_active":"2024-06-14T22:38:42.485155+0000","last_became_peered":"2024-06-14T22:38:42.485155+0000","last_unstale":"2024-06-14T22:39:30.317236+0000","last_undegraded":"2024-06-14T22:39:30.317236+0000","last_fullsized":"2024-06-14T22:39:30.317236+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T07:44:39.241782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317386+0000","last_change":"2024-06-14T22:38:43.725781+0000","last_active":"2024-06-14T22:39:30.317386+0000","last_peered":"2024-06-14T22:39:30.317386+0000","last_clean":"2024-06-14T22:39:30.317386+0000","last_became_active":"2024-06-14T22:38:43.725485+0000","last_became_peered":"2024-06-14T22:38:43.725485+0000","last_unstale":"2024-06-14T22:39:30.317386+0000","last_undegraded":"2024-06-14T22:39:30.317386+0000","last_fullsized":"2024-06-14T22:39:30.317386+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:45:51.438501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"69'8","reported_seq":22,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.314309+0000","last_change":"2024-06-14T22:38:42.478299+0000","last_active":"2024-06-14T22:39:30.314309+0000","last_peered":"2024-06-14T22:39:30.314309+0000","last_clean":"2024-06-14T22:39:30.314309+0000","last_became_active":"2024-06-14T22:38:42.478003+0000","last_became_peered":"2024-06-14T22:38:42.478003+0000","last_unstale":"2024-06-14T22:39:30.314309+0000","last_undegraded":"2024-06-14T22:39:30.314309+0000","last_fullsized":"2024-06-14T22:39:30.314309+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T05:42:55.206984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316841+0000","last_change":"2024-06-14T22:38:42.484453+0000","last_active":"2024-06-14T22:39:30.316841+0000","last_peered":"2024-06-14T22:39:30.316841+0000","last_clean":"2024-06-14T22:39:30.316841+0000","last_became_active":"2024-06-14T22:38:42.484197+0000","last_became_peered":"2024-06-14T22:38:42.484197+0000","last_unstale":"2024-06-14T22:39:30.316841+0000","last_undegraded":"2024-06-14T22:39:30.316841+0000","last_fullsized":"2024-06-14T22:39:30.316841+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T03:51:09.192420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312218+0000","last_change":"2024-06-14T22:38:43.224952+0000","last_active":"2024-06-14T22:39:30.312218+0000","last_peered":"2024-06-14T22:39:30.312218+0000","last_clean":"2024-06-14T22:39:30.312218+0000","last_became_active":"2024-06-14T22:38:43.224641+0000","last_became_peered":"2024-06-14T22:38:43.224641+0000","last_unstale":"2024-06-14T22:39:30.312218+0000","last_undegraded":"2024-06-14T22:39:30.312218+0000","last_fullsized":"2024-06-14T22:39:30.312218+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T05:01:12.242058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317000+0000","last_change":"2024-06-14T22:38:42.484376+0000","last_active":"2024-06-14T22:39:30.317000+0000","last_peered":"2024-06-14T22:39:30.317000+0000","last_clean":"2024-06-14T22:39:30.317000+0000","last_became_active":"2024-06-14T22:38:42.484101+0000","last_became_peered":"2024-06-14T22:38:42.484101+0000","last_unstale":"2024-06-14T22:39:30.317000+0000","last_undegraded":"2024-06-14T22:39:30.317000+0000","last_fullsized":"2024-06-14T22:39:30.317000+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T02:13:59.692695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316959+0000","last_change":"2024-06-14T22:38:42.501616+0000","last_active":"2024-06-14T22:39:30.316959+0000","last_peered":"2024-06-14T22:39:30.316959+0000","last_clean":"2024-06-14T22:39:30.316959+0000","last_became_active":"2024-06-14T22:38:42.501349+0000","last_became_peered":"2024-06-14T22:38:42.501349+0000","last_unstale":"2024-06-14T22:39:30.316959+0000","last_undegraded":"2024-06-14T22:39:30.316959+0000","last_fullsized":"2024-06-14T22:39:30.316959+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:41:52.512276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317049+0000","last_change":"2024-06-14T22:38:42.481563+0000","last_active":"2024-06-14T22:39:30.317049+0000","last_peered":"2024-06-14T22:39:30.317049+0000","last_clean":"2024-06-14T22:39:30.317049+0000","last_became_active":"2024-06-14T22:38:42.481280+0000","last_became_peered":"2024-06-14T22:38:42.481280+0000","last_unstale":"2024-06-14T22:39:30.317049+0000","last_undegraded":"2024-06-14T22:39:30.317049+0000","last_fullsized":"2024-06-14T22:39:30.317049+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T07:16:57.785354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-14T22:39:31.320929+0000","last_change":"2024-06-14T22:39:31.320929+0000","last_active":"2024-06-14T22:39:31.320929+0000","last_peered":"2024-06-14T22:39:31.320929+0000","last_clean":"2024-06-14T22:39:31.320929+0000","last_became_active":"2024-06-14T22:39:31.320567+0000","last_became_peered":"2024-06-14T22:39:31.320567+0000","last_unstale":"2024-06-14T22:39:31.320929+0000","last_undegraded":"2024-06-14T22:39:31.320929+0000","last_fullsized":"2024-06-14T22:39:31.320929+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T01:04:24.799701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312581+0000","last_change":"2024-06-14T22:38:42.482077+0000","last_active":"2024-06-14T22:39:30.312581+0000","last_peered":"2024-06-14T22:39:30.312581+0000","last_clean":"2024-06-14T22:39:30.312581+0000","last_became_active":"2024-06-14T22:38:42.481815+0000","last_became_peered":"2024-06-14T22:38:42.481815+0000","last_unstale":"2024-06-14T22:39:30.312581+0000","last_undegraded":"2024-06-14T22:39:30.312581+0000","last_fullsized":"2024-06-14T22:39:30.312581+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T08:18:07.731271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312791+0000","last_change":"2024-06-14T22:38:44.401833+0000","last_active":"2024-06-14T22:39:30.312791+0000","last_peered":"2024-06-14T22:39:30.312791+0000","last_clean":"2024-06-14T22:39:30.312791+0000","last_became_active":"2024-06-14T22:38:44.401536+0000","last_became_peered":"2024-06-14T22:38:44.401536+0000","last_unstale":"2024-06-14T22:39:30.312791+0000","last_undegraded":"2024-06-14T22:39:30.312791+0000","last_fullsized":"2024-06-14T22:39:30.312791+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T02:46:16.249798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316762+0000","last_change":"2024-06-14T22:38:42.503373+0000","last_active":"2024-06-14T22:39:30.316762+0000","last_peered":"2024-06-14T22:39:30.316762+0000","last_clean":"2024-06-14T22:39:30.316762+0000","last_became_active":"2024-06-14T22:38:42.503072+0000","last_became_peered":"2024-06-14T22:38:42.503072+0000","last_unstale":"2024-06-14T22:39:30.316762+0000","last_undegraded":"2024-06-14T22:39:30.316762+0000","last_fullsized":"2024-06-14T22:39:30.316762+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T02:53:44.197143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317716+0000","last_change":"2024-06-14T22:38:42.497143+0000","last_active":"2024-06-14T22:39:30.317716+0000","last_peered":"2024-06-14T22:39:30.317716+0000","last_clean":"2024-06-14T22:39:30.317716+0000","last_became_active":"2024-06-14T22:38:42.496885+0000","last_became_peered":"2024-06-14T22:38:42.496885+0000","last_unstale":"2024-06-14T22:39:30.317716+0000","last_undegraded":"2024-06-14T22:39:30.317716+0000","last_fullsized":"2024-06-14T22:39:30.317716+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T08:12:46.865775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.315322+0000","last_change":"2024-06-14T22:38:42.482753+0000","last_active":"2024-06-14T22:39:30.315322+0000","last_peered":"2024-06-14T22:39:30.315322+0000","last_clean":"2024-06-14T22:39:30.315322+0000","last_became_active":"2024-06-14T22:38:42.482265+0000","last_became_peered":"2024-06-14T22:38:42.482265+0000","last_unstale":"2024-06-14T22:39:30.315322+0000","last_undegraded":"2024-06-14T22:39:30.315322+0000","last_fullsized":"2024-06-14T22:39:30.315322+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:59:37.488322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-14T22:39:31.408846+0000","last_change":"2024-06-14T22:39:31.408846+0000","last_active":"2024-06-14T22:39:31.408846+0000","last_peered":"2024-06-14T22:39:31.408846+0000","last_clean":"2024-06-14T22:39:31.408846+0000","last_became_active":"2024-06-14T22:39:31.408551+0000","last_became_peered":"2024-06-14T22:39:31.408551+0000","last_unstale":"2024-06-14T22:39:31.408846+0000","last_undegraded":"2024-06-14T22:39:31.408846+0000","last_fullsized":"2024-06-14T22:39:31.408846+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T03:45:49.377391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317128+0000","last_change":"2024-06-14T22:38:42.484378+0000","last_active":"2024-06-14T22:39:30.317128+0000","last_peered":"2024-06-14T22:39:30.317128+0000","last_clean":"2024-06-14T22:39:30.317128+0000","last_became_active":"2024-06-14T22:38:42.484029+0000","last_became_peered":"2024-06-14T22:38:42.484029+0000","last_unstale":"2024-06-14T22:39:30.317128+0000","last_undegraded":"2024-06-14T22:39:30.317128+0000","last_fullsized":"2024-06-14T22:39:30.317128+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T08:14:38.230315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317778+0000","last_change":"2024-06-14T22:38:42.498204+0000","last_active":"2024-06-14T22:39:30.317778+0000","last_peered":"2024-06-14T22:39:30.317778+0000","last_clean":"2024-06-14T22:39:30.317778+0000","last_became_active":"2024-06-14T22:38:42.497948+0000","last_became_peered":"2024-06-14T22:38:42.497948+0000","last_unstale":"2024-06-14T22:39:30.317778+0000","last_undegraded":"2024-06-14T22:39:30.317778+0000","last_fullsized":"2024-06-14T22:39:30.317778+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T08:21:42.983849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316369+0000","last_change":"2024-06-14T22:38:42.486827+0000","last_active":"2024-06-14T22:39:30.316369+0000","last_peered":"2024-06-14T22:39:30.316369+0000","last_clean":"2024-06-14T22:39:30.316369+0000","last_became_active":"2024-06-14T22:38:42.486459+0000","last_became_peered":"2024-06-14T22:38:42.486459+0000","last_unstale":"2024-06-14T22:39:30.316369+0000","last_undegraded":"2024-06-14T22:39:30.316369+0000","last_fullsized":"2024-06-14T22:39:30.316369+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:55:32.336101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.314713+0000","last_change":"2024-06-14T22:38:42.500893+0000","last_active":"2024-06-14T22:39:30.314713+0000","last_peered":"2024-06-14T22:39:30.314713+0000","last_clean":"2024-06-14T22:39:30.314713+0000","last_became_active":"2024-06-14T22:38:42.500616+0000","last_became_peered":"2024-06-14T22:38:42.500616+0000","last_unstale":"2024-06-14T22:39:30.314713+0000","last_undegraded":"2024-06-14T22:39:30.314713+0000","last_fullsized":"2024-06-14T22:39:30.314713+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T07:24:07.809972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"63'128","reported_seq":278,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316467+0000","last_change":"2024-06-14T22:37:01.481768+0000","last_active":"2024-06-14T22:39:30.316467+0000","last_peered":"2024-06-14T22:39:30.316467+0000","last_clean":"2024-06-14T22:39:30.316467+0000","last_became_active":"2024-06-14T22:37:01.481339+0000","last_became_peered":"2024-06-14T22:37:01.481339+0000","last_unstale":"2024-06-14T22:39:30.316467+0000","last_undegraded":"2024-06-14T22:39:30.316467+0000","last_fullsized":"2024-06-14T22:39:30.316467+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:34:59.252736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:34:59.252736+0000","last_clean_scrub_stamp":"2024-06-14T22:34:59.252736+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T22:55:57.126904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312349+0000","last_change":"2024-06-14T22:37:31.278821+0000","last_active":"2024-06-14T22:39:30.312349+0000","last_peered":"2024-06-14T22:39:30.312349+0000","last_clean":"2024-06-14T22:39:30.312349+0000","last_became_active":"2024-06-14T22:37:31.278504+0000","last_became_peered":"2024-06-14T22:37:31.278504+0000","last_unstale":"2024-06-14T22:39:30.312349+0000","last_undegraded":"2024-06-14T22:39:30.312349+0000","last_fullsized":"2024-06-14T22:39:30.312349+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T10:00:23.519181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317138+0000","last_change":"2024-06-14T22:38:42.486161+0000","last_active":"2024-06-14T22:39:30.317138+0000","last_peered":"2024-06-14T22:39:30.317138+0000","last_clean":"2024-06-14T22:39:30.317138+0000","last_became_active":"2024-06-14T22:38:42.485877+0000","last_became_peered":"2024-06-14T22:38:42.485877+0000","last_unstale":"2024-06-14T22:39:30.317138+0000","last_undegraded":"2024-06-14T22:39:30.317138+0000","last_fullsized":"2024-06-14T22:39:30.317138+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T01:20:52.720802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.314523+0000","last_change":"2024-06-14T22:38:43.224955+0000","last_active":"2024-06-14T22:39:30.314523+0000","last_peered":"2024-06-14T22:39:30.314523+0000","last_clean":"2024-06-14T22:39:30.314523+0000","last_became_active":"2024-06-14T22:38:43.224728+0000","last_became_peered":"2024-06-14T22:38:43.224728+0000","last_unstale":"2024-06-14T22:39:30.314523+0000","last_undegraded":"2024-06-14T22:39:30.314523+0000","last_fullsized":"2024-06-14T22:39:30.314523+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T05:44:35.552763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316456+0000","last_change":"2024-06-14T22:37:30.199225+0000","last_active":"2024-06-14T22:39:30.316456+0000","last_peered":"2024-06-14T22:39:30.316456+0000","last_clean":"2024-06-14T22:39:30.316456+0000","last_became_active":"2024-06-14T22:37:30.199011+0000","last_became_peered":"2024-06-14T22:37:30.199011+0000","last_unstale":"2024-06-14T22:39:30.316456+0000","last_undegraded":"2024-06-14T22:39:30.316456+0000","last_fullsized":"2024-06-14T22:39:30.316456+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:45:43.859896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312365+0000","last_change":"2024-06-14T22:38:42.488670+0000","last_active":"2024-06-14T22:39:30.312365+0000","last_peered":"2024-06-14T22:39:30.312365+0000","last_clean":"2024-06-14T22:39:30.312365+0000","last_became_active":"2024-06-14T22:38:42.488378+0000","last_became_peered":"2024-06-14T22:38:42.488378+0000","last_unstale":"2024-06-14T22:39:30.312365+0000","last_undegraded":"2024-06-14T22:39:30.312365+0000","last_fullsized":"2024-06-14T22:39:30.312365+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T04:05:55.767676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313321+0000","last_change":"2024-06-14T22:38:43.560986+0000","last_active":"2024-06-14T22:39:30.313321+0000","last_peered":"2024-06-14T22:39:30.313321+0000","last_clean":"2024-06-14T22:39:30.313321+0000","last_became_active":"2024-06-14T22:38:43.560654+0000","last_became_peered":"2024-06-14T22:38:43.560654+0000","last_unstale":"2024-06-14T22:39:30.313321+0000","last_undegraded":"2024-06-14T22:39:30.313321+0000","last_fullsized":"2024-06-14T22:39:30.313321+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T03:42:45.301101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317612+0000","last_change":"2024-06-14T22:37:30.327693+0000","last_active":"2024-06-14T22:39:30.317612+0000","last_peered":"2024-06-14T22:39:30.317612+0000","last_clean":"2024-06-14T22:39:30.317612+0000","last_became_active":"2024-06-14T22:37:30.327313+0000","last_became_peered":"2024-06-14T22:37:30.327313+0000","last_unstale":"2024-06-14T22:39:30.317612+0000","last_undegraded":"2024-06-14T22:39:30.317612+0000","last_fullsized":"2024-06-14T22:39:30.317612+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T00:09:29.434302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312687+0000","last_change":"2024-06-14T22:38:42.483694+0000","last_active":"2024-06-14T22:39:30.312687+0000","last_peered":"2024-06-14T22:39:30.312687+0000","last_clean":"2024-06-14T22:39:30.312687+0000","last_became_active":"2024-06-14T22:38:42.483430+0000","last_became_peered":"2024-06-14T22:38:42.483430+0000","last_unstale":"2024-06-14T22:39:30.312687+0000","last_undegraded":"2024-06-14T22:39:30.312687+0000","last_fullsized":"2024-06-14T22:39:30.312687+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T22:49:15.659117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.314786+0000","last_change":"2024-06-14T22:38:42.501792+0000","last_active":"2024-06-14T22:39:30.314786+0000","last_peered":"2024-06-14T22:39:30.314786+0000","last_clean":"2024-06-14T22:39:30.314786+0000","last_became_active":"2024-06-14T22:38:42.501512+0000","last_became_peered":"2024-06-14T22:38:42.501512+0000","last_unstale":"2024-06-14T22:39:30.314786+0000","last_undegraded":"2024-06-14T22:39:30.314786+0000","last_fullsized":"2024-06-14T22:39:30.314786+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T02:46:29.665939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312321+0000","last_change":"2024-06-14T22:37:31.471372+0000","last_active":"2024-06-14T22:39:30.312321+0000","last_peered":"2024-06-14T22:39:30.312321+0000","last_clean":"2024-06-14T22:39:30.312321+0000","last_became_active":"2024-06-14T22:37:31.471087+0000","last_became_peered":"2024-06-14T22:37:31.471087+0000","last_unstale":"2024-06-14T22:39:30.312321+0000","last_undegraded":"2024-06-14T22:39:30.312321+0000","last_fullsized":"2024-06-14T22:39:30.312321+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:31:03.406918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317802+0000","last_change":"2024-06-14T22:38:42.487113+0000","last_active":"2024-06-14T22:39:30.317802+0000","last_peered":"2024-06-14T22:39:30.317802+0000","last_clean":"2024-06-14T22:39:30.317802+0000","last_became_active":"2024-06-14T22:38:42.486821+0000","last_became_peered":"2024-06-14T22:38:42.486821+0000","last_unstale":"2024-06-14T22:39:30.317802+0000","last_undegraded":"2024-06-14T22:39:30.317802+0000","last_fullsized":"2024-06-14T22:39:30.317802+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T05:08:38.707677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313394+0000","last_change":"2024-06-14T22:38:42.500265+0000","last_active":"2024-06-14T22:39:30.313394+0000","last_peered":"2024-06-14T22:39:30.313394+0000","last_clean":"2024-06-14T22:39:30.313394+0000","last_became_active":"2024-06-14T22:38:42.500008+0000","last_became_peered":"2024-06-14T22:38:42.500008+0000","last_unstale":"2024-06-14T22:39:30.313394+0000","last_undegraded":"2024-06-14T22:39:30.313394+0000","last_fullsized":"2024-06-14T22:39:30.313394+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T05:55:38.641160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312615+0000","last_change":"2024-06-14T22:38:42.484647+0000","last_active":"2024-06-14T22:39:30.312615+0000","last_peered":"2024-06-14T22:39:30.312615+0000","last_clean":"2024-06-14T22:39:30.312615+0000","last_became_active":"2024-06-14T22:38:42.484400+0000","last_became_peered":"2024-06-14T22:38:42.484400+0000","last_unstale":"2024-06-14T22:39:30.312615+0000","last_undegraded":"2024-06-14T22:39:30.312615+0000","last_fullsized":"2024-06-14T22:39:30.312615+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T09:58:50.502219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313687+0000","last_change":"2024-06-14T22:37:30.197518+0000","last_active":"2024-06-14T22:39:30.313687+0000","last_peered":"2024-06-14T22:39:30.313687+0000","last_clean":"2024-06-14T22:39:30.313687+0000","last_became_active":"2024-06-14T22:37:30.197224+0000","last_became_peered":"2024-06-14T22:37:30.197224+0000","last_unstale":"2024-06-14T22:39:30.313687+0000","last_undegraded":"2024-06-14T22:39:30.313687+0000","last_fullsized":"2024-06-14T22:39:30.313687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T03:41:54.860308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312891+0000","last_change":"2024-06-14T22:38:42.498137+0000","last_active":"2024-06-14T22:39:30.312891+0000","last_peered":"2024-06-14T22:39:30.312891+0000","last_clean":"2024-06-14T22:39:30.312891+0000","last_became_active":"2024-06-14T22:38:42.497875+0000","last_became_peered":"2024-06-14T22:38:42.497875+0000","last_unstale":"2024-06-14T22:39:30.312891+0000","last_undegraded":"2024-06-14T22:39:30.312891+0000","last_fullsized":"2024-06-14T22:39:30.312891+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T09:20:49.783080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313537+0000","last_change":"2024-06-14T22:38:42.480396+0000","last_active":"2024-06-14T22:39:30.313537+0000","last_peered":"2024-06-14T22:39:30.313537+0000","last_clean":"2024-06-14T22:39:30.313537+0000","last_became_active":"2024-06-14T22:38:42.480132+0000","last_became_peered":"2024-06-14T22:38:42.480132+0000","last_unstale":"2024-06-14T22:39:30.313537+0000","last_undegraded":"2024-06-14T22:39:30.313537+0000","last_fullsized":"2024-06-14T22:39:30.313537+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:12:39.633215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317132+0000","last_change":"2024-06-14T22:37:30.196859+0000","last_active":"2024-06-14T22:39:30.317132+0000","last_peered":"2024-06-14T22:39:30.317132+0000","last_clean":"2024-06-14T22:39:30.317132+0000","last_became_active":"2024-06-14T22:37:30.196536+0000","last_became_peered":"2024-06-14T22:37:30.196536+0000","last_unstale":"2024-06-14T22:39:30.317132+0000","last_undegraded":"2024-06-14T22:39:30.317132+0000","last_fullsized":"2024-06-14T22:39:30.317132+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:46:59.482280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312882+0000","last_change":"2024-06-14T22:38:43.725601+0000","last_active":"2024-06-14T22:39:30.312882+0000","last_peered":"2024-06-14T22:39:30.312882+0000","last_clean":"2024-06-14T22:39:30.312882+0000","last_became_active":"2024-06-14T22:38:43.725301+0000","last_became_peered":"2024-06-14T22:38:43.725301+0000","last_unstale":"2024-06-14T22:39:30.312882+0000","last_undegraded":"2024-06-14T22:39:30.312882+0000","last_fullsized":"2024-06-14T22:39:30.312882+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T04:28:00.903854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317554+0000","last_change":"2024-06-14T22:38:42.496591+0000","last_active":"2024-06-14T22:39:30.317554+0000","last_peered":"2024-06-14T22:39:30.317554+0000","last_clean":"2024-06-14T22:39:30.317554+0000","last_became_active":"2024-06-14T22:38:42.496044+0000","last_became_peered":"2024-06-14T22:38:42.496044+0000","last_unstale":"2024-06-14T22:39:30.317554+0000","last_undegraded":"2024-06-14T22:39:30.317554+0000","last_fullsized":"2024-06-14T22:39:30.317554+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:46:21.059388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313668+0000","last_change":"2024-06-14T22:37:30.198386+0000","last_active":"2024-06-14T22:39:30.313668+0000","last_peered":"2024-06-14T22:39:30.313668+0000","last_clean":"2024-06-14T22:39:30.313668+0000","last_became_active":"2024-06-14T22:37:30.198073+0000","last_became_peered":"2024-06-14T22:37:30.198073+0000","last_unstale":"2024-06-14T22:39:30.313668+0000","last_undegraded":"2024-06-14T22:39:30.313668+0000","last_fullsized":"2024-06-14T22:39:30.313668+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T00:16:05.703306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316195+0000","last_change":"2024-06-14T22:38:43.726191+0000","last_active":"2024-06-14T22:39:30.316195+0000","last_peered":"2024-06-14T22:39:30.316195+0000","last_clean":"2024-06-14T22:39:30.316195+0000","last_became_active":"2024-06-14T22:38:43.725905+0000","last_became_peered":"2024-06-14T22:38:43.725905+0000","last_unstale":"2024-06-14T22:39:30.316195+0000","last_undegraded":"2024-06-14T22:39:30.316195+0000","last_fullsized":"2024-06-14T22:39:30.316195+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T01:39:22.404035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313507+0000","last_change":"2024-06-14T22:38:42.479188+0000","last_active":"2024-06-14T22:39:30.313507+0000","last_peered":"2024-06-14T22:39:30.313507+0000","last_clean":"2024-06-14T22:39:30.313507+0000","last_became_active":"2024-06-14T22:38:42.478913+0000","last_became_peered":"2024-06-14T22:38:42.478913+0000","last_unstale":"2024-06-14T22:39:30.313507+0000","last_undegraded":"2024-06-14T22:39:30.313507+0000","last_fullsized":"2024-06-14T22:39:30.313507+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T04:06:28.891556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.314607+0000","last_change":"2024-06-14T22:37:30.196467+0000","last_active":"2024-06-14T22:39:30.314607+0000","last_peered":"2024-06-14T22:39:30.314607+0000","last_clean":"2024-06-14T22:39:30.314607+0000","last_became_active":"2024-06-14T22:37:30.196165+0000","last_became_peered":"2024-06-14T22:37:30.196165+0000","last_unstale":"2024-06-14T22:39:30.314607+0000","last_undegraded":"2024-06-14T22:39:30.314607+0000","last_fullsized":"2024-06-14T22:39:30.314607+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:27:43.756983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317055+0000","last_change":"2024-06-14T22:38:42.494575+0000","last_active":"2024-06-14T22:39:30.317055+0000","last_peered":"2024-06-14T22:39:30.317055+0000","last_clean":"2024-06-14T22:39:30.317055+0000","last_became_active":"2024-06-14T22:38:42.494271+0000","last_became_peered":"2024-06-14T22:38:42.494271+0000","last_unstale":"2024-06-14T22:39:30.317055+0000","last_undegraded":"2024-06-14T22:39:30.317055+0000","last_fullsized":"2024-06-14T22:39:30.317055+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:49:45.930328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317657+0000","last_change":"2024-06-14T22:38:42.501626+0000","last_active":"2024-06-14T22:39:30.317657+0000","last_peered":"2024-06-14T22:39:30.317657+0000","last_clean":"2024-06-14T22:39:30.317657+0000","last_became_active":"2024-06-14T22:38:42.501339+0000","last_became_peered":"2024-06-14T22:38:42.501339+0000","last_unstale":"2024-06-14T22:39:30.317657+0000","last_undegraded":"2024-06-14T22:39:30.317657+0000","last_fullsized":"2024-06-14T22:39:30.317657+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T09:36:57.992534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"69'6","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317829+0000","last_change":"2024-06-14T22:38:42.488111+0000","last_active":"2024-06-14T22:39:30.317829+0000","last_peered":"2024-06-14T22:39:30.317829+0000","last_clean":"2024-06-14T22:39:30.317829+0000","last_became_active":"2024-06-14T22:38:42.487412+0000","last_became_peered":"2024-06-14T22:38:42.487412+0000","last_unstale":"2024-06-14T22:39:30.317829+0000","last_undegraded":"2024-06-14T22:39:30.317829+0000","last_fullsized":"2024-06-14T22:39:30.317829+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T04:24:19.762392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313060+0000","last_change":"2024-06-14T22:38:42.495956+0000","last_active":"2024-06-14T22:39:30.313060+0000","last_peered":"2024-06-14T22:39:30.313060+0000","last_clean":"2024-06-14T22:39:30.313060+0000","last_became_active":"2024-06-14T22:38:42.495688+0000","last_became_peered":"2024-06-14T22:38:42.495688+0000","last_unstale":"2024-06-14T22:39:30.313060+0000","last_undegraded":"2024-06-14T22:39:30.313060+0000","last_fullsized":"2024-06-14T22:39:30.313060+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T03:11:49.264678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"69'3","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316901+0000","last_change":"2024-06-14T22:38:42.485581+0000","last_active":"2024-06-14T22:39:30.316901+0000","last_peered":"2024-06-14T22:39:30.316901+0000","last_clean":"2024-06-14T22:39:30.316901+0000","last_became_active":"2024-06-14T22:38:42.485301+0000","last_became_peered":"2024-06-14T22:38:42.485301+0000","last_unstale":"2024-06-14T22:39:30.316901+0000","last_undegraded":"2024-06-14T22:39:30.316901+0000","last_fullsized":"2024-06-14T22:39:30.316901+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T01:32:22.143436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312450+0000","last_change":"2024-06-14T22:38:42.481358+0000","last_active":"2024-06-14T22:39:30.312450+0000","last_peered":"2024-06-14T22:39:30.312450+0000","last_clean":"2024-06-14T22:39:30.312450+0000","last_became_active":"2024-06-14T22:38:42.480720+0000","last_became_peered":"2024-06-14T22:38:42.480720+0000","last_unstale":"2024-06-14T22:39:30.312450+0000","last_undegraded":"2024-06-14T22:39:30.312450+0000","last_fullsized":"2024-06-14T22:39:30.312450+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T05:55:35.140240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316183+0000","last_change":"2024-06-14T22:38:43.725602+0000","last_active":"2024-06-14T22:39:30.316183+0000","last_peered":"2024-06-14T22:39:30.316183+0000","last_clean":"2024-06-14T22:39:30.316183+0000","last_became_active":"2024-06-14T22:38:43.725310+0000","last_became_peered":"2024-06-14T22:38:43.725310+0000","last_unstale":"2024-06-14T22:39:30.316183+0000","last_undegraded":"2024-06-14T22:39:30.316183+0000","last_fullsized":"2024-06-14T22:39:30.316183+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T04:48:24.226308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313958+0000","last_change":"2024-06-14T22:38:42.489433+0000","last_active":"2024-06-14T22:39:30.313958+0000","last_peered":"2024-06-14T22:39:30.313958+0000","last_clean":"2024-06-14T22:39:30.313958+0000","last_became_active":"2024-06-14T22:38:42.488987+0000","last_became_peered":"2024-06-14T22:38:42.488987+0000","last_unstale":"2024-06-14T22:39:30.313958+0000","last_undegraded":"2024-06-14T22:39:30.313958+0000","last_fullsized":"2024-06-14T22:39:30.313958+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:01:36.948076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316116+0000","last_change":"2024-06-14T22:38:43.516311+0000","last_active":"2024-06-14T22:39:30.316116+0000","last_peered":"2024-06-14T22:39:30.316116+0000","last_clean":"2024-06-14T22:39:30.316116+0000","last_became_active":"2024-06-14T22:38:43.515882+0000","last_became_peered":"2024-06-14T22:38:43.515882+0000","last_unstale":"2024-06-14T22:39:30.316116+0000","last_undegraded":"2024-06-14T22:39:30.316116+0000","last_fullsized":"2024-06-14T22:39:30.316116+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T03:33:34.065205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.314603+0000","last_change":"2024-06-14T22:38:42.496800+0000","last_active":"2024-06-14T22:39:30.314603+0000","last_peered":"2024-06-14T22:39:30.314603+0000","last_clean":"2024-06-14T22:39:30.314603+0000","last_became_active":"2024-06-14T22:38:42.496504+0000","last_became_peered":"2024-06-14T22:38:42.496504+0000","last_unstale":"2024-06-14T22:39:30.314603+0000","last_undegraded":"2024-06-14T22:39:30.314603+0000","last_fullsized":"2024-06-14T22:39:30.314603+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T22:47:38.379031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316295+0000","last_change":"2024-06-14T22:38:42.503210+0000","last_active":"2024-06-14T22:39:30.316295+0000","last_peered":"2024-06-14T22:39:30.316295+0000","last_clean":"2024-06-14T22:39:30.316295+0000","last_became_active":"2024-06-14T22:38:42.502928+0000","last_became_peered":"2024-06-14T22:38:42.502928+0000","last_unstale":"2024-06-14T22:39:30.316295+0000","last_undegraded":"2024-06-14T22:39:30.316295+0000","last_fullsized":"2024-06-14T22:39:30.316295+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T09:26:22.639881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313224+0000","last_change":"2024-06-14T22:38:43.725763+0000","last_active":"2024-06-14T22:39:30.313224+0000","last_peered":"2024-06-14T22:39:30.313224+0000","last_clean":"2024-06-14T22:39:30.313224+0000","last_became_active":"2024-06-14T22:38:43.725485+0000","last_became_peered":"2024-06-14T22:38:43.725485+0000","last_unstale":"2024-06-14T22:39:30.313224+0000","last_undegraded":"2024-06-14T22:39:30.313224+0000","last_fullsized":"2024-06-14T22:39:30.313224+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T00:24:24.011635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312370+0000","last_change":"2024-06-14T22:38:42.482508+0000","last_active":"2024-06-14T22:39:30.312370+0000","last_peered":"2024-06-14T22:39:30.312370+0000","last_clean":"2024-06-14T22:39:30.312370+0000","last_became_active":"2024-06-14T22:38:42.482227+0000","last_became_peered":"2024-06-14T22:38:42.482227+0000","last_unstale":"2024-06-14T22:39:30.312370+0000","last_undegraded":"2024-06-14T22:39:30.312370+0000","last_fullsized":"2024-06-14T22:39:30.312370+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T01:40:01.655850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-14T22:39:31.320972+0000","last_change":"2024-06-14T22:39:31.320972+0000","last_active":"2024-06-14T22:39:31.320972+0000","last_peered":"2024-06-14T22:39:31.320972+0000","last_clean":"2024-06-14T22:39:31.320972+0000","last_became_active":"2024-06-14T22:39:31.320551+0000","last_became_peered":"2024-06-14T22:39:31.320551+0000","last_unstale":"2024-06-14T22:39:31.320972+0000","last_undegraded":"2024-06-14T22:39:31.320972+0000","last_fullsized":"2024-06-14T22:39:31.320972+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T04:35:41.263880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317371+0000","last_change":"2024-06-14T22:38:42.479811+0000","last_active":"2024-06-14T22:39:30.317371+0000","last_peered":"2024-06-14T22:39:30.317371+0000","last_clean":"2024-06-14T22:39:30.317371+0000","last_became_active":"2024-06-14T22:38:42.479572+0000","last_became_peered":"2024-06-14T22:38:42.479572+0000","last_unstale":"2024-06-14T22:39:30.317371+0000","last_undegraded":"2024-06-14T22:39:30.317371+0000","last_fullsized":"2024-06-14T22:39:30.317371+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T01:18:13.593924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316671+0000","last_change":"2024-06-14T22:38:42.498558+0000","last_active":"2024-06-14T22:39:30.316671+0000","last_peered":"2024-06-14T22:39:30.316671+0000","last_clean":"2024-06-14T22:39:30.316671+0000","last_became_active":"2024-06-14T22:38:42.498186+0000","last_became_peered":"2024-06-14T22:38:42.498186+0000","last_unstale":"2024-06-14T22:39:30.316671+0000","last_undegraded":"2024-06-14T22:39:30.316671+0000","last_fullsized":"2024-06-14T22:39:30.316671+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:59:45.354757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313046+0000","last_change":"2024-06-14T22:38:42.480838+0000","last_active":"2024-06-14T22:39:30.313046+0000","last_peered":"2024-06-14T22:39:30.313046+0000","last_clean":"2024-06-14T22:39:30.313046+0000","last_became_active":"2024-06-14T22:38:42.480545+0000","last_became_peered":"2024-06-14T22:38:42.480545+0000","last_unstale":"2024-06-14T22:39:30.313046+0000","last_undegraded":"2024-06-14T22:39:30.313046+0000","last_fullsized":"2024-06-14T22:39:30.313046+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T04:24:24.044224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317685+0000","last_change":"2024-06-14T22:38:42.500697+0000","last_active":"2024-06-14T22:39:30.317685+0000","last_peered":"2024-06-14T22:39:30.317685+0000","last_clean":"2024-06-14T22:39:30.317685+0000","last_became_active":"2024-06-14T22:38:42.500523+0000","last_became_peered":"2024-06-14T22:38:42.500523+0000","last_unstale":"2024-06-14T22:39:30.317685+0000","last_undegraded":"2024-06-14T22:39:30.317685+0000","last_fullsized":"2024-06-14T22:39:30.317685+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T05:33:14.736922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312453+0000","last_change":"2024-06-14T22:38:42.479789+0000","last_active":"2024-06-14T22:39:30.312453+0000","last_peered":"2024-06-14T22:39:30.312453+0000","last_clean":"2024-06-14T22:39:30.312453+0000","last_became_active":"2024-06-14T22:38:42.479384+0000","last_became_peered":"2024-06-14T22:38:42.479384+0000","last_unstale":"2024-06-14T22:39:30.312453+0000","last_undegraded":"2024-06-14T22:39:30.312453+0000","last_fullsized":"2024-06-14T22:39:30.312453+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T03:41:42.884206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317235+0000","last_change":"2024-06-14T22:38:42.500262+0000","last_active":"2024-06-14T22:39:30.317235+0000","last_peered":"2024-06-14T22:39:30.317235+0000","last_clean":"2024-06-14T22:39:30.317235+0000","last_became_active":"2024-06-14T22:38:42.500022+0000","last_became_peered":"2024-06-14T22:38:42.500022+0000","last_unstale":"2024-06-14T22:39:30.317235+0000","last_undegraded":"2024-06-14T22:39:30.317235+0000","last_fullsized":"2024-06-14T22:39:30.317235+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:13:36.394902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312517+0000","last_change":"2024-06-14T22:38:42.483023+0000","last_active":"2024-06-14T22:39:30.312517+0000","last_peered":"2024-06-14T22:39:30.312517+0000","last_clean":"2024-06-14T22:39:30.312517+0000","last_became_active":"2024-06-14T22:38:42.482755+0000","last_became_peered":"2024-06-14T22:38:42.482755+0000","last_unstale":"2024-06-14T22:39:30.312517+0000","last_undegraded":"2024-06-14T22:39:30.312517+0000","last_fullsized":"2024-06-14T22:39:30.312517+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T09:31:30.836640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316387+0000","last_change":"2024-06-14T22:38:43.561035+0000","last_active":"2024-06-14T22:39:30.316387+0000","last_peered":"2024-06-14T22:39:30.316387+0000","last_clean":"2024-06-14T22:39:30.316387+0000","last_became_active":"2024-06-14T22:38:43.560690+0000","last_became_peered":"2024-06-14T22:38:43.560690+0000","last_unstale":"2024-06-14T22:39:30.316387+0000","last_undegraded":"2024-06-14T22:39:30.316387+0000","last_fullsized":"2024-06-14T22:39:30.316387+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T02:23:42.187256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317425+0000","last_change":"2024-06-14T22:38:42.484168+0000","last_active":"2024-06-14T22:39:30.317425+0000","last_peered":"2024-06-14T22:39:30.317425+0000","last_clean":"2024-06-14T22:39:30.317425+0000","last_became_active":"2024-06-14T22:38:42.483890+0000","last_became_peered":"2024-06-14T22:38:42.483890+0000","last_unstale":"2024-06-14T22:39:30.317425+0000","last_undegraded":"2024-06-14T22:39:30.317425+0000","last_fullsized":"2024-06-14T22:39:30.317425+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T10:28:06.461394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316804+0000","last_change":"2024-06-14T22:38:42.498132+0000","last_active":"2024-06-14T22:39:30.316804+0000","last_peered":"2024-06-14T22:39:30.316804+0000","last_clean":"2024-06-14T22:39:30.316804+0000","last_became_active":"2024-06-14T22:38:42.497792+0000","last_became_peered":"2024-06-14T22:38:42.497792+0000","last_unstale":"2024-06-14T22:39:30.316804+0000","last_undegraded":"2024-06-14T22:39:30.316804+0000","last_fullsized":"2024-06-14T22:39:30.316804+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:42:43.598822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"69'4","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.314500+0000","last_change":"2024-06-14T22:38:42.489243+0000","last_active":"2024-06-14T22:39:30.314500+0000","last_peered":"2024-06-14T22:39:30.314500+0000","last_clean":"2024-06-14T22:39:30.314500+0000","last_became_active":"2024-06-14T22:38:42.488151+0000","last_became_peered":"2024-06-14T22:38:42.488151+0000","last_unstale":"2024-06-14T22:39:30.314500+0000","last_undegraded":"2024-06-14T22:39:30.314500+0000","last_fullsized":"2024-06-14T22:39:30.314500+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T22:55:15.550627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":7014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201315,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298668,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446484,"statfs":{"total":95995035648,"available":95689199616,"internally_reserved":0,"allocated":1339392,"data_stored":908101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.496,"15min":0.49099999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48099999999999998,"15min":0.47899999999999998},"min":{"1min":0.38100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.63300000000000001}]},{"osd":1,"last update":"Fri Jun 14 22:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.499,"15min":0.499},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.745,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.39700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.48099999999999998}]},{"osd":2,"last update":"Fri Jun 14 22:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.63300000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.496,"15min":0.49399999999999999},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72599999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.57499999999999996}]},{"osd":3,"last update":"Fri Jun 14 22:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76500000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.501},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70799999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.59799999999999998}]},{"osd":4,"last update":"Fri Jun 14 22:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.42699999999999999,"15min":0.42899999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.68600000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.60799999999999998}]},{"osd":5,"last update":"Fri Jun 14 22:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46300000000000002,"15min":0.46600000000000003},"min":{"1min":0.28899999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67000000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70999999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.51800000000000002}]},{"osd":6,"last update":"Fri Jun 14 22:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.51000000000000001,"15min":0.51300000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.67600000000000005,"5min":0.872,"15min":0.872},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.73199999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.623}]}]},{"osd":6,"up_from":47,"seq":201863462953,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298648,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446504,"statfs":{"total":95995035648,"available":95689220096,"internally_reserved":0,"allocated":1323008,"data_stored":907690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:38:41 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69399999999999995,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.40400000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.66600000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.52200000000000002}]},{"osd":1,"last update":"Fri Jun 14 22:38:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.51000000000000001,"15min":0.51400000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71299999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.63},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.32800000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.53900000000000003}]},{"osd":2,"last update":"Fri Jun 14 22:38:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.503},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.57399999999999995}]},{"osd":3,"last update":"Fri Jun 14 22:38:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.38500000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.61399999999999999}]},{"osd":4,"last update":"Fri Jun 14 22:38:41 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.29499999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40999999999999998,"15min":0.40699999999999997},"min":{"1min":0.29799999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.47599999999999998}]},{"osd":5,"last update":"Fri Jun 14 22:38:41 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.40899999999999997,"15min":0.40300000000000002},"min":{"1min":0.33800000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.437,"15min":0.435},"min":{"1min":0.318,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.55000000000000004}]},{"osd":7,"last update":"Fri Jun 14 22:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.75600000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.505,"15min":0.505},"min":{"1min":0.36899999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.65400000000000003}]}]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298076,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447076,"statfs":{"total":95995035648,"available":95689805824,"internally_reserved":0,"allocated":733184,"data_stored":317193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.64200000000000002,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.63900000000000001,"5min":0.873,"15min":0.873},"last":0.48999999999999999}]},{"osd":1,"last update":"Fri Jun 14 22:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.60099999999999998,"5min":0.748,"15min":0.748},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.58399999999999996}]},{"osd":2,"last update":"Fri Jun 14 22:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.52400000000000002,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.497,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.38200000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.60899999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.55600000000000005}]},{"osd":3,"last update":"Fri Jun 14 22:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71899999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.38700000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.66100000000000003,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.60499999999999998}]},{"osd":4,"last update":"Fri Jun 14 22:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.38200000000000001},"min":{"1min":0.25900000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.56100000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.41299999999999998,"15min":0.39700000000000002},"min":{"1min":0.308,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.59299999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.47099999999999997}]},{"osd":6,"last update":"Fri Jun 14 22:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.46899999999999997,"15min":0.47299999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.63600000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.436},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.44600000000000001,"15min":0.438},"min":{"1min":0.29299999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68799999999999994,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.65400000000000003}]},{"osd":7,"last update":"Fri Jun 14 22:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53700000000000003,"15min":0.54300000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71399999999999997,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.5,"5min":0.52300000000000002,"15min":0.52700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.67800000000000005,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.59599999999999997}]}]},{"osd":4,"up_from":31,"seq":133143986225,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298084,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447068,"statfs":{"total":95995035648,"available":95689797632,"internally_reserved":0,"allocated":745472,"data_stored":317986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56100000000000005,"15min":0.57899999999999996},"min":{"1min":0.379,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72999999999999998,"5min":1.026,"15min":1.026},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54000000000000004,"15min":0.55600000000000005},"min":{"1min":0.38500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68200000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65400000000000003}]},{"osd":1,"last update":"Fri Jun 14 22:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52200000000000002,"15min":0.52700000000000002},"min":{"1min":0.36399999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.753,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56699999999999995,"15min":0.58599999999999997},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69399999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.70299999999999996}]},{"osd":2,"last update":"Fri Jun 14 22:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.378,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77400000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.628},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54600000000000004,"15min":0.55400000000000005},"min":{"1min":0.39100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.71299999999999997,"5min":0.873,"15min":0.873},"last":0.71299999999999997}]},{"osd":3,"last update":"Fri Jun 14 22:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57199999999999995,"15min":0.58199999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81999999999999995,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.57799999999999996,"15min":0.60599999999999998},"min":{"1min":0.38300000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83799999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.64300000000000002}]},{"osd":5,"last update":"Fri Jun 14 22:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.39200000000000002,"15min":0.36699999999999999},"min":{"1min":0.30499999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.40000000000000002,"15min":0.378},"min":{"1min":0.29699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.47499999999999998}]},{"osd":6,"last update":"Fri Jun 14 22:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.41199999999999998,"15min":0.40100000000000002},"min":{"1min":0.33100000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.42999999999999999,"15min":0.41699999999999998},"min":{"1min":0.30099999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.67100000000000004}]},{"osd":7,"last update":"Fri Jun 14 22:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.47299999999999998,"15min":0.45700000000000002},"min":{"1min":0.41799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.45900000000000002,"15min":0.44800000000000001},"min":{"1min":0.40000000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.68600000000000005}]}]},{"osd":3,"up_from":26,"seq":111669149750,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298072,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447080,"statfs":{"total":95995035648,"available":95689809920,"internally_reserved":0,"allocated":729088,"data_stored":316643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:39:30 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38800000000000001,"15min":0.39800000000000002},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.49099999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.441},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.40999999999999998,"15min":0.42599999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.33000000000000002}]},{"osd":1,"last update":"Fri Jun 14 22:39:30 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.27000000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.56299999999999994,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.40899999999999997,"15min":0.41599999999999998},"min":{"1min":0.26800000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.505,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.39400000000000002}]},{"osd":2,"last update":"Fri Jun 14 22:39:30 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.438,"15min":0.45000000000000001},"min":{"1min":0.318,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.621,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.42699999999999999,"15min":0.433},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.57999999999999996,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.36699999999999999}]},{"osd":4,"last update":"Fri Jun 14 22:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.378,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79800000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57099999999999995,"15min":0.56100000000000005},"min":{"1min":0.45500000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.745,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.82299999999999995}]},{"osd":5,"last update":"Fri Jun 14 22:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57099999999999995,"15min":0.58099999999999996},"min":{"1min":0.44400000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73399999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.40100000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.76200000000000001,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.69799999999999995}]},{"osd":6,"last update":"Fri Jun 14 22:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.61799999999999999,"15min":0.626},"min":{"1min":0.40300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.88800000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.874},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.82199999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.81299999999999994}]},{"osd":7,"last update":"Fri Jun 14 22:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.64200000000000002,"15min":0.65100000000000002},"min":{"1min":0.45100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72899999999999998,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.64600000000000002,"15min":0.65400000000000003},"min":{"1min":0.47799999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76100000000000001,"5min":1.599,"15min":1.599},"last":0.67400000000000004}]}]},{"osd":2,"up_from":18,"seq":77309411387,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298076,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447076,"statfs":{"total":95995035648,"available":95689805824,"internally_reserved":0,"allocated":733184,"data_stored":317157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:39:04 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.376,"15min":0.38},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.59099999999999997,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.38600000000000001,"15min":0.39100000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.34399999999999997}]},{"osd":1,"last update":"Fri Jun 14 22:39:04 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.38700000000000001,"15min":0.38500000000000001},"min":{"1min":0.245,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53400000000000003,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.40100000000000002,"15min":0.39500000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.29499999999999998}]},{"osd":3,"last update":"Fri Jun 14 22:39:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.438,"15min":0.45100000000000001},"min":{"1min":0.27800000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.56799999999999995,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.435,"15min":0.46000000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.5,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.35899999999999999}]},{"osd":4,"last update":"Fri Jun 14 22:38:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.70599999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.72599999999999998,"5min":0.876,"15min":0.876},"last":0.505}]},{"osd":5,"last update":"Fri Jun 14 22:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.38,"5min":0.312,"15min":0.312},"max":{"1min":0.65900000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60999999999999999,"15min":0.61699999999999999},"min":{"1min":0.41099999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77300000000000002,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.59799999999999998}]},{"osd":6,"last update":"Fri Jun 14 22:38:35 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.65700000000000003,"15min":0.67000000000000004},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.78100000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.64200000000000002,"15min":0.65000000000000002},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74199999999999999,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54200000000000004}]},{"osd":7,"last update":"Fri Jun 14 22:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.41499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77400000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.626,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77000000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.56299999999999994}]}]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298064,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447088,"statfs":{"total":95995035648,"available":95689818112,"internally_reserved":0,"allocated":724992,"data_stored":316553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:38:38 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34799999999999998,"15min":0.33300000000000002},"min":{"1min":0.214,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.52600000000000002,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.373,"15min":0.34799999999999998},"min":{"1min":0.23100000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.58399999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.52000000000000002}]},{"osd":2,"last update":"Fri Jun 14 22:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42099999999999999,"15min":0.41699999999999998},"min":{"1min":0.26800000000000002,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.57199999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.373,"5min":0.38600000000000001,"15min":0.38900000000000001},"min":{"1min":0.252,"5min":0.246,"15min":0.246},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.44900000000000001}]},{"osd":3,"last update":"Fri Jun 14 22:39:31 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40500000000000003,"15min":0.40899999999999997},"min":{"1min":0.247,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61899999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.45600000000000002,"15min":0.46100000000000002},"min":{"1min":0.30399999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.627,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.627}]},{"osd":4,"last update":"Fri Jun 14 22:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.40300000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72999999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.42699999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.748,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.69599999999999995}]},{"osd":5,"last update":"Fri Jun 14 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60099999999999998,"15min":0.61099999999999999},"min":{"1min":0.41999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.78200000000000003,"5min":1.032,"15min":1.032},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58099999999999996,"15min":0.59099999999999997},"min":{"1min":0.375,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91600000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.54300000000000004}]},{"osd":6,"last update":"Fri Jun 14 22:38:36 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.58599999999999997,"15min":0.58199999999999996},"min":{"1min":0.42999999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.60299999999999998,"15min":0.60099999999999998},"min":{"1min":0.44800000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.85399999999999998,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.80800000000000005}]},{"osd":7,"last update":"Fri Jun 14 22:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.39400000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.50900000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72899999999999998,"5min":1.056,"15min":1.056},"last":0.86299999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298644,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446508,"statfs":{"total":95995035648,"available":95689224192,"internally_reserved":0,"allocated":1318912,"data_stored":906983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 14 22:38:41 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.317},"min":{"1min":0.24099999999999999,"5min":0.214,"15min":0.214},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.372,"15min":0.33100000000000002},"min":{"1min":0.28399999999999997,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.28699999999999998}]},{"osd":2,"last update":"Fri Jun 14 22:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.40500000000000003,"15min":0.376},"min":{"1min":0.27000000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.58799999999999997,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39800000000000002,"15min":0.36599999999999999},"min":{"1min":0.26900000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.91200000000000003,"5min":1.51,"15min":1.51},"last":0.32000000000000001}]},{"osd":3,"last update":"Fri Jun 14 22:39:29 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.40200000000000002,"15min":0.41599999999999998},"min":{"1min":0.25,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64800000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.46500000000000002,"15min":0.47899999999999998},"min":{"1min":0.32300000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.57299999999999995,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.41199999999999998}]},{"osd":4,"last update":"Fri Jun 14 22:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61099999999999999,"15min":0.61199999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.53800000000000003}]},{"osd":5,"last update":"Fri Jun 14 22:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57099999999999995,"15min":0.57999999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73299999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.499},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60999999999999999,"15min":0.60699999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.88700000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.46899999999999997}]},{"osd":6,"last update":"Fri Jun 14 22:38:37 2024","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.61599999999999999,"15min":0.61199999999999999},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60699999999999998,"15min":0.60099999999999998},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.51700000000000002}]},{"osd":7,"last update":"Fri Jun 14 22:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.64500000000000002,"15min":0.65500000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84099999999999997,"5min":1.673,"15min":1.673},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.45800000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.745,"5min":0.874,"15min":0.874},"last":0.44}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-14T22:39:40.540 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph pg dump --format=json 2024-06-14T22:39:40.806 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:41.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:41 smithi046 ceph-mon[29586]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-14T22:39:41.443 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:41 smithi046 ceph-mon[29586]: pgmap v78: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:41.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:41 smithi046 ceph-mon[38286]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-14T22:39:41.444 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:41 smithi046 ceph-mon[38286]: pgmap v78: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:41.444 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:41 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:39:41] "GET /metrics HTTP/1.1" 200 239073 "" "Prometheus/2.43.0" 2024-06-14T22:39:41.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:41 smithi178 ceph-mon[32942]: from='client.14739 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-14T22:39:41.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:41 smithi178 ceph-mon[32942]: pgmap v78: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:42.593 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:39:42.594 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-06-14T22:39:43.342 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2024-06-14T22:39:42.223771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592706,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":224,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":167,"ondisk_log_size":167,"up":219,"acting":219,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":219,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2386332,"kb_used_data":7468,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747574884,"statfs":{"total":767960285184,"available":765516681216,"internally_reserved":0,"allocated":7647232,"data_stored":4308306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.907830"},"pg_stats":[{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316868+0000","last_change":"2024-06-14T22:38:43.725693+0000","last_active":"2024-06-14T22:39:30.316868+0000","last_peered":"2024-06-14T22:39:30.316868+0000","last_clean":"2024-06-14T22:39:30.316868+0000","last_became_active":"2024-06-14T22:38:43.725411+0000","last_became_peered":"2024-06-14T22:38:43.725411+0000","last_unstale":"2024-06-14T22:39:30.316868+0000","last_undegraded":"2024-06-14T22:39:30.316868+0000","last_fullsized":"2024-06-14T22:39:30.316868+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T00:22:31.533026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316562+0000","last_change":"2024-06-14T22:38:42.485033+0000","last_active":"2024-06-14T22:39:30.316562+0000","last_peered":"2024-06-14T22:39:30.316562+0000","last_clean":"2024-06-14T22:39:30.316562+0000","last_became_active":"2024-06-14T22:38:42.484143+0000","last_became_peered":"2024-06-14T22:38:42.484143+0000","last_unstale":"2024-06-14T22:39:30.316562+0000","last_undegraded":"2024-06-14T22:39:30.316562+0000","last_fullsized":"2024-06-14T22:39:30.316562+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:44:59.321972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312519+0000","last_change":"2024-06-14T22:38:43.226317+0000","last_active":"2024-06-14T22:39:30.312519+0000","last_peered":"2024-06-14T22:39:30.312519+0000","last_clean":"2024-06-14T22:39:30.312519+0000","last_became_active":"2024-06-14T22:38:43.226041+0000","last_became_peered":"2024-06-14T22:38:43.226041+0000","last_unstale":"2024-06-14T22:39:30.312519+0000","last_undegraded":"2024-06-14T22:39:30.312519+0000","last_fullsized":"2024-06-14T22:39:30.312519+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T09:01:07.074548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312628+0000","last_change":"2024-06-14T22:38:42.489752+0000","last_active":"2024-06-14T22:39:30.312628+0000","last_peered":"2024-06-14T22:39:30.312628+0000","last_clean":"2024-06-14T22:39:30.312628+0000","last_became_active":"2024-06-14T22:38:42.489474+0000","last_became_peered":"2024-06-14T22:38:42.489474+0000","last_unstale":"2024-06-14T22:39:30.312628+0000","last_undegraded":"2024-06-14T22:39:30.312628+0000","last_fullsized":"2024-06-14T22:39:30.312628+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T03:03:35.515139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313406+0000","last_change":"2024-06-14T22:38:43.726372+0000","last_active":"2024-06-14T22:39:30.313406+0000","last_peered":"2024-06-14T22:39:30.313406+0000","last_clean":"2024-06-14T22:39:30.313406+0000","last_became_active":"2024-06-14T22:38:43.726078+0000","last_became_peered":"2024-06-14T22:38:43.726078+0000","last_unstale":"2024-06-14T22:39:30.313406+0000","last_undegraded":"2024-06-14T22:39:30.313406+0000","last_fullsized":"2024-06-14T22:39:30.313406+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:05:21.767046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317236+0000","last_change":"2024-06-14T22:38:42.485551+0000","last_active":"2024-06-14T22:39:30.317236+0000","last_peered":"2024-06-14T22:39:30.317236+0000","last_clean":"2024-06-14T22:39:30.317236+0000","last_became_active":"2024-06-14T22:38:42.485155+0000","last_became_peered":"2024-06-14T22:38:42.485155+0000","last_unstale":"2024-06-14T22:39:30.317236+0000","last_undegraded":"2024-06-14T22:39:30.317236+0000","last_fullsized":"2024-06-14T22:39:30.317236+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T07:44:39.241782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317386+0000","last_change":"2024-06-14T22:38:43.725781+0000","last_active":"2024-06-14T22:39:30.317386+0000","last_peered":"2024-06-14T22:39:30.317386+0000","last_clean":"2024-06-14T22:39:30.317386+0000","last_became_active":"2024-06-14T22:38:43.725485+0000","last_became_peered":"2024-06-14T22:38:43.725485+0000","last_unstale":"2024-06-14T22:39:30.317386+0000","last_undegraded":"2024-06-14T22:39:30.317386+0000","last_fullsized":"2024-06-14T22:39:30.317386+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:45:51.438501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"69'8","reported_seq":22,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.314309+0000","last_change":"2024-06-14T22:38:42.478299+0000","last_active":"2024-06-14T22:39:30.314309+0000","last_peered":"2024-06-14T22:39:30.314309+0000","last_clean":"2024-06-14T22:39:30.314309+0000","last_became_active":"2024-06-14T22:38:42.478003+0000","last_became_peered":"2024-06-14T22:38:42.478003+0000","last_unstale":"2024-06-14T22:39:30.314309+0000","last_undegraded":"2024-06-14T22:39:30.314309+0000","last_fullsized":"2024-06-14T22:39:30.314309+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T05:42:55.206984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316841+0000","last_change":"2024-06-14T22:38:42.484453+0000","last_active":"2024-06-14T22:39:30.316841+0000","last_peered":"2024-06-14T22:39:30.316841+0000","last_clean":"2024-06-14T22:39:30.316841+0000","last_became_active":"2024-06-14T22:38:42.484197+0000","last_became_peered":"2024-06-14T22:38:42.484197+0000","last_unstale":"2024-06-14T22:39:30.316841+0000","last_undegraded":"2024-06-14T22:39:30.316841+0000","last_fullsized":"2024-06-14T22:39:30.316841+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T03:51:09.192420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312218+0000","last_change":"2024-06-14T22:38:43.224952+0000","last_active":"2024-06-14T22:39:30.312218+0000","last_peered":"2024-06-14T22:39:30.312218+0000","last_clean":"2024-06-14T22:39:30.312218+0000","last_became_active":"2024-06-14T22:38:43.224641+0000","last_became_peered":"2024-06-14T22:38:43.224641+0000","last_unstale":"2024-06-14T22:39:30.312218+0000","last_undegraded":"2024-06-14T22:39:30.312218+0000","last_fullsized":"2024-06-14T22:39:30.312218+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T05:01:12.242058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317000+0000","last_change":"2024-06-14T22:38:42.484376+0000","last_active":"2024-06-14T22:39:30.317000+0000","last_peered":"2024-06-14T22:39:30.317000+0000","last_clean":"2024-06-14T22:39:30.317000+0000","last_became_active":"2024-06-14T22:38:42.484101+0000","last_became_peered":"2024-06-14T22:38:42.484101+0000","last_unstale":"2024-06-14T22:39:30.317000+0000","last_undegraded":"2024-06-14T22:39:30.317000+0000","last_fullsized":"2024-06-14T22:39:30.317000+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T02:13:59.692695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316959+0000","last_change":"2024-06-14T22:38:42.501616+0000","last_active":"2024-06-14T22:39:30.316959+0000","last_peered":"2024-06-14T22:39:30.316959+0000","last_clean":"2024-06-14T22:39:30.316959+0000","last_became_active":"2024-06-14T22:38:42.501349+0000","last_became_peered":"2024-06-14T22:38:42.501349+0000","last_unstale":"2024-06-14T22:39:30.316959+0000","last_undegraded":"2024-06-14T22:39:30.316959+0000","last_fullsized":"2024-06-14T22:39:30.316959+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:41:52.512276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317049+0000","last_change":"2024-06-14T22:38:42.481563+0000","last_active":"2024-06-14T22:39:30.317049+0000","last_peered":"2024-06-14T22:39:30.317049+0000","last_clean":"2024-06-14T22:39:30.317049+0000","last_became_active":"2024-06-14T22:38:42.481280+0000","last_became_peered":"2024-06-14T22:38:42.481280+0000","last_unstale":"2024-06-14T22:39:30.317049+0000","last_undegraded":"2024-06-14T22:39:30.317049+0000","last_fullsized":"2024-06-14T22:39:30.317049+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T07:16:57.785354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-14T22:39:31.320929+0000","last_change":"2024-06-14T22:39:31.320929+0000","last_active":"2024-06-14T22:39:31.320929+0000","last_peered":"2024-06-14T22:39:31.320929+0000","last_clean":"2024-06-14T22:39:31.320929+0000","last_became_active":"2024-06-14T22:39:31.320567+0000","last_became_peered":"2024-06-14T22:39:31.320567+0000","last_unstale":"2024-06-14T22:39:31.320929+0000","last_undegraded":"2024-06-14T22:39:31.320929+0000","last_fullsized":"2024-06-14T22:39:31.320929+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T01:04:24.799701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312581+0000","last_change":"2024-06-14T22:38:42.482077+0000","last_active":"2024-06-14T22:39:30.312581+0000","last_peered":"2024-06-14T22:39:30.312581+0000","last_clean":"2024-06-14T22:39:30.312581+0000","last_became_active":"2024-06-14T22:38:42.481815+0000","last_became_peered":"2024-06-14T22:38:42.481815+0000","last_unstale":"2024-06-14T22:39:30.312581+0000","last_undegraded":"2024-06-14T22:39:30.312581+0000","last_fullsized":"2024-06-14T22:39:30.312581+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T08:18:07.731271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312791+0000","last_change":"2024-06-14T22:38:44.401833+0000","last_active":"2024-06-14T22:39:30.312791+0000","last_peered":"2024-06-14T22:39:30.312791+0000","last_clean":"2024-06-14T22:39:30.312791+0000","last_became_active":"2024-06-14T22:38:44.401536+0000","last_became_peered":"2024-06-14T22:38:44.401536+0000","last_unstale":"2024-06-14T22:39:30.312791+0000","last_undegraded":"2024-06-14T22:39:30.312791+0000","last_fullsized":"2024-06-14T22:39:30.312791+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T02:46:16.249798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316762+0000","last_change":"2024-06-14T22:38:42.503373+0000","last_active":"2024-06-14T22:39:30.316762+0000","last_peered":"2024-06-14T22:39:30.316762+0000","last_clean":"2024-06-14T22:39:30.316762+0000","last_became_active":"2024-06-14T22:38:42.503072+0000","last_became_peered":"2024-06-14T22:38:42.503072+0000","last_unstale":"2024-06-14T22:39:30.316762+0000","last_undegraded":"2024-06-14T22:39:30.316762+0000","last_fullsized":"2024-06-14T22:39:30.316762+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T02:53:44.197143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317716+0000","last_change":"2024-06-14T22:38:42.497143+0000","last_active":"2024-06-14T22:39:30.317716+0000","last_peered":"2024-06-14T22:39:30.317716+0000","last_clean":"2024-06-14T22:39:30.317716+0000","last_became_active":"2024-06-14T22:38:42.496885+0000","last_became_peered":"2024-06-14T22:38:42.496885+0000","last_unstale":"2024-06-14T22:39:30.317716+0000","last_undegraded":"2024-06-14T22:39:30.317716+0000","last_fullsized":"2024-06-14T22:39:30.317716+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T08:12:46.865775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.315322+0000","last_change":"2024-06-14T22:38:42.482753+0000","last_active":"2024-06-14T22:39:30.315322+0000","last_peered":"2024-06-14T22:39:30.315322+0000","last_clean":"2024-06-14T22:39:30.315322+0000","last_became_active":"2024-06-14T22:38:42.482265+0000","last_became_peered":"2024-06-14T22:38:42.482265+0000","last_unstale":"2024-06-14T22:39:30.315322+0000","last_undegraded":"2024-06-14T22:39:30.315322+0000","last_fullsized":"2024-06-14T22:39:30.315322+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:59:37.488322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-14T22:39:31.408846+0000","last_change":"2024-06-14T22:39:31.408846+0000","last_active":"2024-06-14T22:39:31.408846+0000","last_peered":"2024-06-14T22:39:31.408846+0000","last_clean":"2024-06-14T22:39:31.408846+0000","last_became_active":"2024-06-14T22:39:31.408551+0000","last_became_peered":"2024-06-14T22:39:31.408551+0000","last_unstale":"2024-06-14T22:39:31.408846+0000","last_undegraded":"2024-06-14T22:39:31.408846+0000","last_fullsized":"2024-06-14T22:39:31.408846+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T03:45:49.377391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317128+0000","last_change":"2024-06-14T22:38:42.484378+0000","last_active":"2024-06-14T22:39:30.317128+0000","last_peered":"2024-06-14T22:39:30.317128+0000","last_clean":"2024-06-14T22:39:30.317128+0000","last_became_active":"2024-06-14T22:38:42.484029+0000","last_became_peered":"2024-06-14T22:38:42.484029+0000","last_unstale":"2024-06-14T22:39:30.317128+0000","last_undegraded":"2024-06-14T22:39:30.317128+0000","last_fullsized":"2024-06-14T22:39:30.317128+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T08:14:38.230315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317778+0000","last_change":"2024-06-14T22:38:42.498204+0000","last_active":"2024-06-14T22:39:30.317778+0000","last_peered":"2024-06-14T22:39:30.317778+0000","last_clean":"2024-06-14T22:39:30.317778+0000","last_became_active":"2024-06-14T22:38:42.497948+0000","last_became_peered":"2024-06-14T22:38:42.497948+0000","last_unstale":"2024-06-14T22:39:30.317778+0000","last_undegraded":"2024-06-14T22:39:30.317778+0000","last_fullsized":"2024-06-14T22:39:30.317778+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T08:21:42.983849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316369+0000","last_change":"2024-06-14T22:38:42.486827+0000","last_active":"2024-06-14T22:39:30.316369+0000","last_peered":"2024-06-14T22:39:30.316369+0000","last_clean":"2024-06-14T22:39:30.316369+0000","last_became_active":"2024-06-14T22:38:42.486459+0000","last_became_peered":"2024-06-14T22:38:42.486459+0000","last_unstale":"2024-06-14T22:39:30.316369+0000","last_undegraded":"2024-06-14T22:39:30.316369+0000","last_fullsized":"2024-06-14T22:39:30.316369+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:55:32.336101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.314713+0000","last_change":"2024-06-14T22:38:42.500893+0000","last_active":"2024-06-14T22:39:30.314713+0000","last_peered":"2024-06-14T22:39:30.314713+0000","last_clean":"2024-06-14T22:39:30.314713+0000","last_became_active":"2024-06-14T22:38:42.500616+0000","last_became_peered":"2024-06-14T22:38:42.500616+0000","last_unstale":"2024-06-14T22:39:30.314713+0000","last_undegraded":"2024-06-14T22:39:30.314713+0000","last_fullsized":"2024-06-14T22:39:30.314713+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T07:24:07.809972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"63'128","reported_seq":278,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316467+0000","last_change":"2024-06-14T22:37:01.481768+0000","last_active":"2024-06-14T22:39:30.316467+0000","last_peered":"2024-06-14T22:39:30.316467+0000","last_clean":"2024-06-14T22:39:30.316467+0000","last_became_active":"2024-06-14T22:37:01.481339+0000","last_became_peered":"2024-06-14T22:37:01.481339+0000","last_unstale":"2024-06-14T22:39:30.316467+0000","last_undegraded":"2024-06-14T22:39:30.316467+0000","last_fullsized":"2024-06-14T22:39:30.316467+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:34:59.252736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:34:59.252736+0000","last_clean_scrub_stamp":"2024-06-14T22:34:59.252736+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T22:55:57.126904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312349+0000","last_change":"2024-06-14T22:37:31.278821+0000","last_active":"2024-06-14T22:39:30.312349+0000","last_peered":"2024-06-14T22:39:30.312349+0000","last_clean":"2024-06-14T22:39:30.312349+0000","last_became_active":"2024-06-14T22:37:31.278504+0000","last_became_peered":"2024-06-14T22:37:31.278504+0000","last_unstale":"2024-06-14T22:39:30.312349+0000","last_undegraded":"2024-06-14T22:39:30.312349+0000","last_fullsized":"2024-06-14T22:39:30.312349+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T10:00:23.519181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317138+0000","last_change":"2024-06-14T22:38:42.486161+0000","last_active":"2024-06-14T22:39:30.317138+0000","last_peered":"2024-06-14T22:39:30.317138+0000","last_clean":"2024-06-14T22:39:30.317138+0000","last_became_active":"2024-06-14T22:38:42.485877+0000","last_became_peered":"2024-06-14T22:38:42.485877+0000","last_unstale":"2024-06-14T22:39:30.317138+0000","last_undegraded":"2024-06-14T22:39:30.317138+0000","last_fullsized":"2024-06-14T22:39:30.317138+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T01:20:52.720802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.314523+0000","last_change":"2024-06-14T22:38:43.224955+0000","last_active":"2024-06-14T22:39:30.314523+0000","last_peered":"2024-06-14T22:39:30.314523+0000","last_clean":"2024-06-14T22:39:30.314523+0000","last_became_active":"2024-06-14T22:38:43.224728+0000","last_became_peered":"2024-06-14T22:38:43.224728+0000","last_unstale":"2024-06-14T22:39:30.314523+0000","last_undegraded":"2024-06-14T22:39:30.314523+0000","last_fullsized":"2024-06-14T22:39:30.314523+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T05:44:35.552763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316456+0000","last_change":"2024-06-14T22:37:30.199225+0000","last_active":"2024-06-14T22:39:30.316456+0000","last_peered":"2024-06-14T22:39:30.316456+0000","last_clean":"2024-06-14T22:39:30.316456+0000","last_became_active":"2024-06-14T22:37:30.199011+0000","last_became_peered":"2024-06-14T22:37:30.199011+0000","last_unstale":"2024-06-14T22:39:30.316456+0000","last_undegraded":"2024-06-14T22:39:30.316456+0000","last_fullsized":"2024-06-14T22:39:30.316456+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:45:43.859896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312365+0000","last_change":"2024-06-14T22:38:42.488670+0000","last_active":"2024-06-14T22:39:30.312365+0000","last_peered":"2024-06-14T22:39:30.312365+0000","last_clean":"2024-06-14T22:39:30.312365+0000","last_became_active":"2024-06-14T22:38:42.488378+0000","last_became_peered":"2024-06-14T22:38:42.488378+0000","last_unstale":"2024-06-14T22:39:30.312365+0000","last_undegraded":"2024-06-14T22:39:30.312365+0000","last_fullsized":"2024-06-14T22:39:30.312365+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T04:05:55.767676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313321+0000","last_change":"2024-06-14T22:38:43.560986+0000","last_active":"2024-06-14T22:39:30.313321+0000","last_peered":"2024-06-14T22:39:30.313321+0000","last_clean":"2024-06-14T22:39:30.313321+0000","last_became_active":"2024-06-14T22:38:43.560654+0000","last_became_peered":"2024-06-14T22:38:43.560654+0000","last_unstale":"2024-06-14T22:39:30.313321+0000","last_undegraded":"2024-06-14T22:39:30.313321+0000","last_fullsized":"2024-06-14T22:39:30.313321+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T03:42:45.301101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317612+0000","last_change":"2024-06-14T22:37:30.327693+0000","last_active":"2024-06-14T22:39:30.317612+0000","last_peered":"2024-06-14T22:39:30.317612+0000","last_clean":"2024-06-14T22:39:30.317612+0000","last_became_active":"2024-06-14T22:37:30.327313+0000","last_became_peered":"2024-06-14T22:37:30.327313+0000","last_unstale":"2024-06-14T22:39:30.317612+0000","last_undegraded":"2024-06-14T22:39:30.317612+0000","last_fullsized":"2024-06-14T22:39:30.317612+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T00:09:29.434302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312687+0000","last_change":"2024-06-14T22:38:42.483694+0000","last_active":"2024-06-14T22:39:30.312687+0000","last_peered":"2024-06-14T22:39:30.312687+0000","last_clean":"2024-06-14T22:39:30.312687+0000","last_became_active":"2024-06-14T22:38:42.483430+0000","last_became_peered":"2024-06-14T22:38:42.483430+0000","last_unstale":"2024-06-14T22:39:30.312687+0000","last_undegraded":"2024-06-14T22:39:30.312687+0000","last_fullsized":"2024-06-14T22:39:30.312687+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T22:49:15.659117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.314786+0000","last_change":"2024-06-14T22:38:42.501792+0000","last_active":"2024-06-14T22:39:30.314786+0000","last_peered":"2024-06-14T22:39:30.314786+0000","last_clean":"2024-06-14T22:39:30.314786+0000","last_became_active":"2024-06-14T22:38:42.501512+0000","last_became_peered":"2024-06-14T22:38:42.501512+0000","last_unstale":"2024-06-14T22:39:30.314786+0000","last_undegraded":"2024-06-14T22:39:30.314786+0000","last_fullsized":"2024-06-14T22:39:30.314786+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T02:46:29.665939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312321+0000","last_change":"2024-06-14T22:37:31.471372+0000","last_active":"2024-06-14T22:39:30.312321+0000","last_peered":"2024-06-14T22:39:30.312321+0000","last_clean":"2024-06-14T22:39:30.312321+0000","last_became_active":"2024-06-14T22:37:31.471087+0000","last_became_peered":"2024-06-14T22:37:31.471087+0000","last_unstale":"2024-06-14T22:39:30.312321+0000","last_undegraded":"2024-06-14T22:39:30.312321+0000","last_fullsized":"2024-06-14T22:39:30.312321+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:31:03.406918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317802+0000","last_change":"2024-06-14T22:38:42.487113+0000","last_active":"2024-06-14T22:39:30.317802+0000","last_peered":"2024-06-14T22:39:30.317802+0000","last_clean":"2024-06-14T22:39:30.317802+0000","last_became_active":"2024-06-14T22:38:42.486821+0000","last_became_peered":"2024-06-14T22:38:42.486821+0000","last_unstale":"2024-06-14T22:39:30.317802+0000","last_undegraded":"2024-06-14T22:39:30.317802+0000","last_fullsized":"2024-06-14T22:39:30.317802+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T05:08:38.707677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313394+0000","last_change":"2024-06-14T22:38:42.500265+0000","last_active":"2024-06-14T22:39:30.313394+0000","last_peered":"2024-06-14T22:39:30.313394+0000","last_clean":"2024-06-14T22:39:30.313394+0000","last_became_active":"2024-06-14T22:38:42.500008+0000","last_became_peered":"2024-06-14T22:38:42.500008+0000","last_unstale":"2024-06-14T22:39:30.313394+0000","last_undegraded":"2024-06-14T22:39:30.313394+0000","last_fullsized":"2024-06-14T22:39:30.313394+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T05:55:38.641160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312615+0000","last_change":"2024-06-14T22:38:42.484647+0000","last_active":"2024-06-14T22:39:30.312615+0000","last_peered":"2024-06-14T22:39:30.312615+0000","last_clean":"2024-06-14T22:39:30.312615+0000","last_became_active":"2024-06-14T22:38:42.484400+0000","last_became_peered":"2024-06-14T22:38:42.484400+0000","last_unstale":"2024-06-14T22:39:30.312615+0000","last_undegraded":"2024-06-14T22:39:30.312615+0000","last_fullsized":"2024-06-14T22:39:30.312615+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T09:58:50.502219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313687+0000","last_change":"2024-06-14T22:37:30.197518+0000","last_active":"2024-06-14T22:39:30.313687+0000","last_peered":"2024-06-14T22:39:30.313687+0000","last_clean":"2024-06-14T22:39:30.313687+0000","last_became_active":"2024-06-14T22:37:30.197224+0000","last_became_peered":"2024-06-14T22:37:30.197224+0000","last_unstale":"2024-06-14T22:39:30.313687+0000","last_undegraded":"2024-06-14T22:39:30.313687+0000","last_fullsized":"2024-06-14T22:39:30.313687+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T03:41:54.860308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312891+0000","last_change":"2024-06-14T22:38:42.498137+0000","last_active":"2024-06-14T22:39:30.312891+0000","last_peered":"2024-06-14T22:39:30.312891+0000","last_clean":"2024-06-14T22:39:30.312891+0000","last_became_active":"2024-06-14T22:38:42.497875+0000","last_became_peered":"2024-06-14T22:38:42.497875+0000","last_unstale":"2024-06-14T22:39:30.312891+0000","last_undegraded":"2024-06-14T22:39:30.312891+0000","last_fullsized":"2024-06-14T22:39:30.312891+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T09:20:49.783080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313537+0000","last_change":"2024-06-14T22:38:42.480396+0000","last_active":"2024-06-14T22:39:30.313537+0000","last_peered":"2024-06-14T22:39:30.313537+0000","last_clean":"2024-06-14T22:39:30.313537+0000","last_became_active":"2024-06-14T22:38:42.480132+0000","last_became_peered":"2024-06-14T22:38:42.480132+0000","last_unstale":"2024-06-14T22:39:30.313537+0000","last_undegraded":"2024-06-14T22:39:30.313537+0000","last_fullsized":"2024-06-14T22:39:30.313537+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:12:39.633215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317132+0000","last_change":"2024-06-14T22:37:30.196859+0000","last_active":"2024-06-14T22:39:30.317132+0000","last_peered":"2024-06-14T22:39:30.317132+0000","last_clean":"2024-06-14T22:39:30.317132+0000","last_became_active":"2024-06-14T22:37:30.196536+0000","last_became_peered":"2024-06-14T22:37:30.196536+0000","last_unstale":"2024-06-14T22:39:30.317132+0000","last_undegraded":"2024-06-14T22:39:30.317132+0000","last_fullsized":"2024-06-14T22:39:30.317132+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:46:59.482280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312882+0000","last_change":"2024-06-14T22:38:43.725601+0000","last_active":"2024-06-14T22:39:30.312882+0000","last_peered":"2024-06-14T22:39:30.312882+0000","last_clean":"2024-06-14T22:39:30.312882+0000","last_became_active":"2024-06-14T22:38:43.725301+0000","last_became_peered":"2024-06-14T22:38:43.725301+0000","last_unstale":"2024-06-14T22:39:30.312882+0000","last_undegraded":"2024-06-14T22:39:30.312882+0000","last_fullsized":"2024-06-14T22:39:30.312882+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T04:28:00.903854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317554+0000","last_change":"2024-06-14T22:38:42.496591+0000","last_active":"2024-06-14T22:39:30.317554+0000","last_peered":"2024-06-14T22:39:30.317554+0000","last_clean":"2024-06-14T22:39:30.317554+0000","last_became_active":"2024-06-14T22:38:42.496044+0000","last_became_peered":"2024-06-14T22:38:42.496044+0000","last_unstale":"2024-06-14T22:39:30.317554+0000","last_undegraded":"2024-06-14T22:39:30.317554+0000","last_fullsized":"2024-06-14T22:39:30.317554+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:46:21.059388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313668+0000","last_change":"2024-06-14T22:37:30.198386+0000","last_active":"2024-06-14T22:39:30.313668+0000","last_peered":"2024-06-14T22:39:30.313668+0000","last_clean":"2024-06-14T22:39:30.313668+0000","last_became_active":"2024-06-14T22:37:30.198073+0000","last_became_peered":"2024-06-14T22:37:30.198073+0000","last_unstale":"2024-06-14T22:39:30.313668+0000","last_undegraded":"2024-06-14T22:39:30.313668+0000","last_fullsized":"2024-06-14T22:39:30.313668+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T00:16:05.703306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316195+0000","last_change":"2024-06-14T22:38:43.726191+0000","last_active":"2024-06-14T22:39:30.316195+0000","last_peered":"2024-06-14T22:39:30.316195+0000","last_clean":"2024-06-14T22:39:30.316195+0000","last_became_active":"2024-06-14T22:38:43.725905+0000","last_became_peered":"2024-06-14T22:38:43.725905+0000","last_unstale":"2024-06-14T22:39:30.316195+0000","last_undegraded":"2024-06-14T22:39:30.316195+0000","last_fullsized":"2024-06-14T22:39:30.316195+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T01:39:22.404035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313507+0000","last_change":"2024-06-14T22:38:42.479188+0000","last_active":"2024-06-14T22:39:30.313507+0000","last_peered":"2024-06-14T22:39:30.313507+0000","last_clean":"2024-06-14T22:39:30.313507+0000","last_became_active":"2024-06-14T22:38:42.478913+0000","last_became_peered":"2024-06-14T22:38:42.478913+0000","last_unstale":"2024-06-14T22:39:30.313507+0000","last_undegraded":"2024-06-14T22:39:30.313507+0000","last_fullsized":"2024-06-14T22:39:30.313507+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T04:06:28.891556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.314607+0000","last_change":"2024-06-14T22:37:30.196467+0000","last_active":"2024-06-14T22:39:30.314607+0000","last_peered":"2024-06-14T22:39:30.314607+0000","last_clean":"2024-06-14T22:39:30.314607+0000","last_became_active":"2024-06-14T22:37:30.196165+0000","last_became_peered":"2024-06-14T22:37:30.196165+0000","last_unstale":"2024-06-14T22:39:30.314607+0000","last_undegraded":"2024-06-14T22:39:30.314607+0000","last_fullsized":"2024-06-14T22:39:30.314607+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:37:30.156310+0000","last_clean_scrub_stamp":"2024-06-14T22:37:30.156310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:27:43.756983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317055+0000","last_change":"2024-06-14T22:38:42.494575+0000","last_active":"2024-06-14T22:39:30.317055+0000","last_peered":"2024-06-14T22:39:30.317055+0000","last_clean":"2024-06-14T22:39:30.317055+0000","last_became_active":"2024-06-14T22:38:42.494271+0000","last_became_peered":"2024-06-14T22:38:42.494271+0000","last_unstale":"2024-06-14T22:39:30.317055+0000","last_undegraded":"2024-06-14T22:39:30.317055+0000","last_fullsized":"2024-06-14T22:39:30.317055+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:49:45.930328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317657+0000","last_change":"2024-06-14T22:38:42.501626+0000","last_active":"2024-06-14T22:39:30.317657+0000","last_peered":"2024-06-14T22:39:30.317657+0000","last_clean":"2024-06-14T22:39:30.317657+0000","last_became_active":"2024-06-14T22:38:42.501339+0000","last_became_peered":"2024-06-14T22:38:42.501339+0000","last_unstale":"2024-06-14T22:39:30.317657+0000","last_undegraded":"2024-06-14T22:39:30.317657+0000","last_fullsized":"2024-06-14T22:39:30.317657+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T09:36:57.992534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"69'6","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317829+0000","last_change":"2024-06-14T22:38:42.488111+0000","last_active":"2024-06-14T22:39:30.317829+0000","last_peered":"2024-06-14T22:39:30.317829+0000","last_clean":"2024-06-14T22:39:30.317829+0000","last_became_active":"2024-06-14T22:38:42.487412+0000","last_became_peered":"2024-06-14T22:38:42.487412+0000","last_unstale":"2024-06-14T22:39:30.317829+0000","last_undegraded":"2024-06-14T22:39:30.317829+0000","last_fullsized":"2024-06-14T22:39:30.317829+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T04:24:19.762392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1552,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313060+0000","last_change":"2024-06-14T22:38:42.495956+0000","last_active":"2024-06-14T22:39:30.313060+0000","last_peered":"2024-06-14T22:39:30.313060+0000","last_clean":"2024-06-14T22:39:30.313060+0000","last_became_active":"2024-06-14T22:38:42.495688+0000","last_became_peered":"2024-06-14T22:38:42.495688+0000","last_unstale":"2024-06-14T22:39:30.313060+0000","last_undegraded":"2024-06-14T22:39:30.313060+0000","last_fullsized":"2024-06-14T22:39:30.313060+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T03:11:49.264678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"69'3","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316901+0000","last_change":"2024-06-14T22:38:42.485581+0000","last_active":"2024-06-14T22:39:30.316901+0000","last_peered":"2024-06-14T22:39:30.316901+0000","last_clean":"2024-06-14T22:39:30.316901+0000","last_became_active":"2024-06-14T22:38:42.485301+0000","last_became_peered":"2024-06-14T22:38:42.485301+0000","last_unstale":"2024-06-14T22:39:30.316901+0000","last_undegraded":"2024-06-14T22:39:30.316901+0000","last_fullsized":"2024-06-14T22:39:30.316901+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T01:32:22.143436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312450+0000","last_change":"2024-06-14T22:38:42.481358+0000","last_active":"2024-06-14T22:39:30.312450+0000","last_peered":"2024-06-14T22:39:30.312450+0000","last_clean":"2024-06-14T22:39:30.312450+0000","last_became_active":"2024-06-14T22:38:42.480720+0000","last_became_peered":"2024-06-14T22:38:42.480720+0000","last_unstale":"2024-06-14T22:39:30.312450+0000","last_undegraded":"2024-06-14T22:39:30.312450+0000","last_fullsized":"2024-06-14T22:39:30.312450+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T05:55:35.140240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316183+0000","last_change":"2024-06-14T22:38:43.725602+0000","last_active":"2024-06-14T22:39:30.316183+0000","last_peered":"2024-06-14T22:39:30.316183+0000","last_clean":"2024-06-14T22:39:30.316183+0000","last_became_active":"2024-06-14T22:38:43.725310+0000","last_became_peered":"2024-06-14T22:38:43.725310+0000","last_unstale":"2024-06-14T22:39:30.316183+0000","last_undegraded":"2024-06-14T22:39:30.316183+0000","last_fullsized":"2024-06-14T22:39:30.316183+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T04:48:24.226308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313958+0000","last_change":"2024-06-14T22:38:42.489433+0000","last_active":"2024-06-14T22:39:30.313958+0000","last_peered":"2024-06-14T22:39:30.313958+0000","last_clean":"2024-06-14T22:39:30.313958+0000","last_became_active":"2024-06-14T22:38:42.488987+0000","last_became_peered":"2024-06-14T22:38:42.488987+0000","last_unstale":"2024-06-14T22:39:30.313958+0000","last_undegraded":"2024-06-14T22:39:30.313958+0000","last_fullsized":"2024-06-14T22:39:30.313958+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:01:36.948076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316116+0000","last_change":"2024-06-14T22:38:43.516311+0000","last_active":"2024-06-14T22:39:30.316116+0000","last_peered":"2024-06-14T22:39:30.316116+0000","last_clean":"2024-06-14T22:39:30.316116+0000","last_became_active":"2024-06-14T22:38:43.515882+0000","last_became_peered":"2024-06-14T22:38:43.515882+0000","last_unstale":"2024-06-14T22:39:30.316116+0000","last_undegraded":"2024-06-14T22:39:30.316116+0000","last_fullsized":"2024-06-14T22:39:30.316116+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T03:33:34.065205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.314603+0000","last_change":"2024-06-14T22:38:42.496800+0000","last_active":"2024-06-14T22:39:30.314603+0000","last_peered":"2024-06-14T22:39:30.314603+0000","last_clean":"2024-06-14T22:39:30.314603+0000","last_became_active":"2024-06-14T22:38:42.496504+0000","last_became_peered":"2024-06-14T22:38:42.496504+0000","last_unstale":"2024-06-14T22:39:30.314603+0000","last_undegraded":"2024-06-14T22:39:30.314603+0000","last_fullsized":"2024-06-14T22:39:30.314603+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T22:47:38.379031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316295+0000","last_change":"2024-06-14T22:38:42.503210+0000","last_active":"2024-06-14T22:39:30.316295+0000","last_peered":"2024-06-14T22:39:30.316295+0000","last_clean":"2024-06-14T22:39:30.316295+0000","last_became_active":"2024-06-14T22:38:42.502928+0000","last_became_peered":"2024-06-14T22:38:42.502928+0000","last_unstale":"2024-06-14T22:39:30.316295+0000","last_undegraded":"2024-06-14T22:39:30.316295+0000","last_fullsized":"2024-06-14T22:39:30.316295+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T09:26:22.639881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313224+0000","last_change":"2024-06-14T22:38:43.725763+0000","last_active":"2024-06-14T22:39:30.313224+0000","last_peered":"2024-06-14T22:39:30.313224+0000","last_clean":"2024-06-14T22:39:30.313224+0000","last_became_active":"2024-06-14T22:38:43.725485+0000","last_became_peered":"2024-06-14T22:38:43.725485+0000","last_unstale":"2024-06-14T22:39:30.313224+0000","last_undegraded":"2024-06-14T22:39:30.313224+0000","last_fullsized":"2024-06-14T22:39:30.313224+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T00:24:24.011635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312370+0000","last_change":"2024-06-14T22:38:42.482508+0000","last_active":"2024-06-14T22:39:30.312370+0000","last_peered":"2024-06-14T22:39:30.312370+0000","last_clean":"2024-06-14T22:39:30.312370+0000","last_became_active":"2024-06-14T22:38:42.482227+0000","last_became_peered":"2024-06-14T22:38:42.482227+0000","last_unstale":"2024-06-14T22:39:30.312370+0000","last_undegraded":"2024-06-14T22:39:30.312370+0000","last_fullsized":"2024-06-14T22:39:30.312370+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T01:40:01.655850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":23,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-06-14T22:39:31.320972+0000","last_change":"2024-06-14T22:39:31.320972+0000","last_active":"2024-06-14T22:39:31.320972+0000","last_peered":"2024-06-14T22:39:31.320972+0000","last_clean":"2024-06-14T22:39:31.320972+0000","last_became_active":"2024-06-14T22:39:31.320551+0000","last_became_peered":"2024-06-14T22:39:31.320551+0000","last_unstale":"2024-06-14T22:39:31.320972+0000","last_undegraded":"2024-06-14T22:39:31.320972+0000","last_fullsized":"2024-06-14T22:39:31.320972+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T04:35:41.263880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317371+0000","last_change":"2024-06-14T22:38:42.479811+0000","last_active":"2024-06-14T22:39:30.317371+0000","last_peered":"2024-06-14T22:39:30.317371+0000","last_clean":"2024-06-14T22:39:30.317371+0000","last_became_active":"2024-06-14T22:38:42.479572+0000","last_became_peered":"2024-06-14T22:38:42.479572+0000","last_unstale":"2024-06-14T22:39:30.317371+0000","last_undegraded":"2024-06-14T22:39:30.317371+0000","last_fullsized":"2024-06-14T22:39:30.317371+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T01:18:13.593924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316671+0000","last_change":"2024-06-14T22:38:42.498558+0000","last_active":"2024-06-14T22:39:30.316671+0000","last_peered":"2024-06-14T22:39:30.316671+0000","last_clean":"2024-06-14T22:39:30.316671+0000","last_became_active":"2024-06-14T22:38:42.498186+0000","last_became_peered":"2024-06-14T22:38:42.498186+0000","last_unstale":"2024-06-14T22:39:30.316671+0000","last_undegraded":"2024-06-14T22:39:30.316671+0000","last_fullsized":"2024-06-14T22:39:30.316671+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:59:45.354757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.313046+0000","last_change":"2024-06-14T22:38:42.480838+0000","last_active":"2024-06-14T22:39:30.313046+0000","last_peered":"2024-06-14T22:39:30.313046+0000","last_clean":"2024-06-14T22:39:30.313046+0000","last_became_active":"2024-06-14T22:38:42.480545+0000","last_became_peered":"2024-06-14T22:38:42.480545+0000","last_unstale":"2024-06-14T22:39:30.313046+0000","last_undegraded":"2024-06-14T22:39:30.313046+0000","last_fullsized":"2024-06-14T22:39:30.313046+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T04:24:24.044224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317685+0000","last_change":"2024-06-14T22:38:42.500697+0000","last_active":"2024-06-14T22:39:30.317685+0000","last_peered":"2024-06-14T22:39:30.317685+0000","last_clean":"2024-06-14T22:39:30.317685+0000","last_became_active":"2024-06-14T22:38:42.500523+0000","last_became_peered":"2024-06-14T22:38:42.500523+0000","last_unstale":"2024-06-14T22:39:30.317685+0000","last_undegraded":"2024-06-14T22:39:30.317685+0000","last_fullsized":"2024-06-14T22:39:30.317685+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T05:33:14.736922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312453+0000","last_change":"2024-06-14T22:38:42.479789+0000","last_active":"2024-06-14T22:39:30.312453+0000","last_peered":"2024-06-14T22:39:30.312453+0000","last_clean":"2024-06-14T22:39:30.312453+0000","last_became_active":"2024-06-14T22:38:42.479384+0000","last_became_peered":"2024-06-14T22:38:42.479384+0000","last_unstale":"2024-06-14T22:39:30.312453+0000","last_undegraded":"2024-06-14T22:39:30.312453+0000","last_fullsized":"2024-06-14T22:39:30.312453+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T03:41:42.884206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317235+0000","last_change":"2024-06-14T22:38:42.500262+0000","last_active":"2024-06-14T22:39:30.317235+0000","last_peered":"2024-06-14T22:39:30.317235+0000","last_clean":"2024-06-14T22:39:30.317235+0000","last_became_active":"2024-06-14T22:38:42.500022+0000","last_became_peered":"2024-06-14T22:38:42.500022+0000","last_unstale":"2024-06-14T22:39:30.317235+0000","last_undegraded":"2024-06-14T22:39:30.317235+0000","last_fullsized":"2024-06-14T22:39:30.317235+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T23:13:36.394902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"69'2","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.312517+0000","last_change":"2024-06-14T22:38:42.483023+0000","last_active":"2024-06-14T22:39:30.312517+0000","last_peered":"2024-06-14T22:39:30.312517+0000","last_clean":"2024-06-14T22:39:30.312517+0000","last_became_active":"2024-06-14T22:38:42.482755+0000","last_became_peered":"2024-06-14T22:38:42.482755+0000","last_unstale":"2024-06-14T22:39:30.312517+0000","last_undegraded":"2024-06-14T22:39:30.312517+0000","last_fullsized":"2024-06-14T22:39:30.312517+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T09:31:30.836640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316387+0000","last_change":"2024-06-14T22:38:43.561035+0000","last_active":"2024-06-14T22:39:30.316387+0000","last_peered":"2024-06-14T22:39:30.316387+0000","last_clean":"2024-06-14T22:39:30.316387+0000","last_became_active":"2024-06-14T22:38:43.560690+0000","last_became_peered":"2024-06-14T22:38:43.560690+0000","last_unstale":"2024-06-14T22:39:30.316387+0000","last_undegraded":"2024-06-14T22:39:30.316387+0000","last_fullsized":"2024-06-14T22:39:30.316387+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T02:23:42.187256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"69'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.317425+0000","last_change":"2024-06-14T22:38:42.484168+0000","last_active":"2024-06-14T22:39:30.317425+0000","last_peered":"2024-06-14T22:39:30.317425+0000","last_clean":"2024-06-14T22:39:30.317425+0000","last_became_active":"2024-06-14T22:38:42.483890+0000","last_became_peered":"2024-06-14T22:38:42.483890+0000","last_unstale":"2024-06-14T22:39:30.317425+0000","last_undegraded":"2024-06-14T22:39:30.317425+0000","last_fullsized":"2024-06-14T22:39:30.317425+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T10:28:06.461394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.316804+0000","last_change":"2024-06-14T22:38:42.498132+0000","last_active":"2024-06-14T22:39:30.316804+0000","last_peered":"2024-06-14T22:39:30.316804+0000","last_clean":"2024-06-14T22:39:30.316804+0000","last_became_active":"2024-06-14T22:38:42.497792+0000","last_became_peered":"2024-06-14T22:38:42.497792+0000","last_unstale":"2024-06-14T22:39:30.316804+0000","last_undegraded":"2024-06-14T22:39:30.316804+0000","last_fullsized":"2024-06-14T22:39:30.316804+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:42.466890+0000","last_clean_scrub_stamp":"2024-06-14T22:38:42.466890+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-16T06:42:43.598822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"69'4","reported_seq":18,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-06-14T22:39:30.314500+0000","last_change":"2024-06-14T22:38:42.489243+0000","last_active":"2024-06-14T22:39:30.314500+0000","last_peered":"2024-06-14T22:39:30.314500+0000","last_clean":"2024-06-14T22:39:30.314500+0000","last_became_active":"2024-06-14T22:38:42.488151+0000","last_became_peered":"2024-06-14T22:38:42.488151+0000","last_unstale":"2024-06-14T22:39:30.314500+0000","last_undegraded":"2024-06-14T22:39:30.314500+0000","last_fullsized":"2024-06-14T22:39:30.314500+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-14T22:38:41.465091+0000","last_clean_scrub_stamp":"2024-06-14T22:38:41.465091+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-15T22:55:15.550627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":570,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2338,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":7014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":24,"acting":24,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201316,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298668,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446484,"statfs":{"total":95995035648,"available":95689199616,"internally_reserved":0,"allocated":1339392,"data_stored":908101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.496,"15min":0.49099999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48099999999999998,"15min":0.47899999999999998},"min":{"1min":0.38100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.629}]},{"osd":1,"last update":"Fri Jun 14 22:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.499,"15min":0.499},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.745,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.499,"15min":0.49299999999999999},"min":{"1min":0.39700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.68600000000000005}]},{"osd":2,"last update":"Fri Jun 14 22:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.48299999999999998,"15min":0.48099999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.63300000000000001,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.496,"15min":0.49399999999999999},"min":{"1min":0.36299999999999999,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.72599999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.63700000000000001}]},{"osd":3,"last update":"Fri Jun 14 22:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76500000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54000000000000004,"15min":0.53800000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.70799999999999996,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.65000000000000002}]},{"osd":4,"last update":"Fri Jun 14 22:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.42699999999999999,"15min":0.42899999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.68600000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.42399999999999999}]},{"osd":5,"last update":"Fri Jun 14 22:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.46300000000000002,"15min":0.46600000000000003},"min":{"1min":0.28899999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67000000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.70999999999999996,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.48599999999999999}]},{"osd":6,"last update":"Fri Jun 14 22:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.51000000000000001,"15min":0.51300000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.67600000000000005,"5min":0.872,"15min":0.872},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.73199999999999998,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.61099999999999999}]}]},{"osd":6,"up_from":47,"seq":201863462954,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298648,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446504,"statfs":{"total":95995035648,"available":95689220096,"internally_reserved":0,"allocated":1323008,"data_stored":907690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:38:41 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.53100000000000003,"15min":0.53700000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.69399999999999995,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.40400000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.66600000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.503}]},{"osd":1,"last update":"Fri Jun 14 22:38:41 2024","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.51000000000000001,"15min":0.51400000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71299999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.32800000000000001,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.442}]},{"osd":2,"last update":"Fri Jun 14 22:38:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55000000000000004,"15min":0.54900000000000004},"min":{"1min":0.38800000000000001,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Jun 14 22:38:41 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.52700000000000002,"15min":0.52300000000000002},"min":{"1min":0.38500000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.60299999999999998}]},{"osd":4,"last update":"Fri Jun 14 22:38:41 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.40400000000000003,"15min":0.40200000000000002},"min":{"1min":0.29499999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.40999999999999998,"15min":0.40699999999999997},"min":{"1min":0.29799999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Jun 14 22:38:41 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.40899999999999997,"15min":0.40300000000000002},"min":{"1min":0.33800000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.437,"15min":0.435},"min":{"1min":0.318,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.36399999999999999}]},{"osd":7,"last update":"Fri Jun 14 22:39:02 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.75600000000000001,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.505,"15min":0.505},"min":{"1min":0.36899999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.56699999999999995}]}]},{"osd":5,"up_from":39,"seq":167503724590,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298076,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447076,"statfs":{"total":95995035648,"available":95689805824,"internally_reserved":0,"allocated":733184,"data_stored":317193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.52900000000000003,"15min":0.53300000000000003},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.64200000000000002,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.63900000000000001,"5min":0.873,"15min":0.873},"last":0.56899999999999995}]},{"osd":1,"last update":"Fri Jun 14 22:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.502,"15min":0.49399999999999999},"min":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.60099999999999998,"5min":0.748,"15min":0.748},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.53900000000000003,"15min":0.52300000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.432}]},{"osd":2,"last update":"Fri Jun 14 22:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.52400000000000002,"15min":0.50900000000000001},"min":{"1min":0.373,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.38200000000000001,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.60899999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.44800000000000001}]},{"osd":3,"last update":"Fri Jun 14 22:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.52900000000000003,"15min":0.52700000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71899999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.38700000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.66100000000000003,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.48099999999999998}]},{"osd":4,"last update":"Fri Jun 14 22:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.38200000000000001},"min":{"1min":0.25900000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.56100000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.438,"5min":0.41299999999999998,"15min":0.39700000000000002},"min":{"1min":0.308,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.59299999999999997,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.71999999999999997}]},{"osd":6,"last update":"Fri Jun 14 22:39:38 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46800000000000003,"15min":0.47199999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.64400000000000002,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.46600000000000003,"15min":0.44500000000000001},"min":{"1min":0.32700000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66800000000000004,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.63500000000000001}]},{"osd":7,"last update":"Fri Jun 14 22:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53700000000000003,"15min":0.54300000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.71399999999999997,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.5,"5min":0.52300000000000002,"15min":0.52700000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.67800000000000005,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.69199999999999995}]}]},{"osd":4,"up_from":31,"seq":133143986226,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298084,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447068,"statfs":{"total":95995035648,"available":95689797632,"internally_reserved":0,"allocated":745472,"data_stored":317986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56100000000000005,"15min":0.57899999999999996},"min":{"1min":0.379,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72999999999999998,"5min":1.026,"15min":1.026},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54000000000000004,"15min":0.55600000000000005},"min":{"1min":0.38500000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.68200000000000005,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.65800000000000003}]},{"osd":1,"last update":"Fri Jun 14 22:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52200000000000002,"15min":0.52700000000000002},"min":{"1min":0.36399999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.753,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.56699999999999995,"15min":0.58599999999999997},"min":{"1min":0.372,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.69399999999999995,"5min":0.95799999999999996,"15min":0.95799999999999996},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Jun 14 22:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53700000000000003,"15min":0.53500000000000003},"min":{"1min":0.378,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.77400000000000002,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54600000000000004,"15min":0.55400000000000005},"min":{"1min":0.39100000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.71299999999999997,"5min":0.873,"15min":0.873},"last":0.57999999999999996}]},{"osd":3,"last update":"Fri Jun 14 22:38:49 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57199999999999995,"15min":0.58199999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.81999999999999995,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.57799999999999996,"15min":0.60599999999999998},"min":{"1min":0.38300000000000001,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.83799999999999997,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.46600000000000003}]},{"osd":5,"last update":"Fri Jun 14 22:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.39200000000000002,"15min":0.36699999999999999},"min":{"1min":0.30499999999999999,"5min":0.252,"15min":0.252},"max":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"last":0.501},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.40000000000000002,"15min":0.378},"min":{"1min":0.29699999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.52200000000000002}]},{"osd":6,"last update":"Fri Jun 14 22:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.42899999999999999,"15min":0.40699999999999997},"min":{"1min":0.33600000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68300000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.44500000000000001,"15min":0.42199999999999999},"min":{"1min":0.33400000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.67100000000000004,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.59499999999999997}]},{"osd":7,"last update":"Fri Jun 14 22:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.47299999999999998,"15min":0.45700000000000002},"min":{"1min":0.41799999999999998,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.45900000000000002,"15min":0.44800000000000001},"min":{"1min":0.40000000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.60699999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149751,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298072,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447080,"statfs":{"total":95995035648,"available":95689809920,"internally_reserved":0,"allocated":729088,"data_stored":316643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:39:30 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.38800000000000001,"15min":0.39800000000000002},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.49099999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.40999999999999998,"15min":0.42599999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.28699999999999998}]},{"osd":1,"last update":"Fri Jun 14 22:39:30 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.27000000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.56299999999999994,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.40899999999999997,"15min":0.41599999999999998},"min":{"1min":0.26800000000000002,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.505,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.26600000000000001}]},{"osd":2,"last update":"Fri Jun 14 22:39:30 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.438,"15min":0.45000000000000001},"min":{"1min":0.318,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.621,"5min":0.66500000000000004,"15min":0.66500000000000004},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.42699999999999999,"15min":0.433},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.57999999999999996,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.315}]},{"osd":4,"last update":"Fri Jun 14 22:38:55 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57099999999999995,"15min":0.56499999999999995},"min":{"1min":0.378,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79800000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57099999999999995,"15min":0.56100000000000005},"min":{"1min":0.45500000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.745,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.61499999999999999}]},{"osd":5,"last update":"Fri Jun 14 22:39:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.57099999999999995,"15min":0.58099999999999996},"min":{"1min":0.44400000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.73399999999999999,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59099999999999997,"15min":0.59599999999999997},"min":{"1min":0.40100000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.76200000000000001,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.48599999999999999}]},{"osd":6,"last update":"Fri Jun 14 22:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.61199999999999999,"15min":0.624},"min":{"1min":0.41999999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.874,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.60299999999999998,"15min":0.59899999999999998},"min":{"1min":0.38600000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81699999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.624}]},{"osd":7,"last update":"Fri Jun 14 22:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.64200000000000002,"15min":0.65100000000000002},"min":{"1min":0.45100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.72899999999999998,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.64600000000000002,"15min":0.65400000000000003},"min":{"1min":0.47799999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.76100000000000001,"5min":1.599,"15min":1.599},"last":0.52900000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298076,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447076,"statfs":{"total":95995035648,"available":95689805824,"internally_reserved":0,"allocated":733184,"data_stored":317157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:39:04 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.376,"15min":0.38},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.59099999999999997,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.36499999999999999,"5min":0.38600000000000001,"15min":0.39100000000000001},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"last":0.30599999999999999}]},{"osd":1,"last update":"Fri Jun 14 22:39:04 2024","interfaces":[{"interface":"back","average":{"1min":0.35999999999999999,"5min":0.38700000000000001,"15min":0.38500000000000001},"min":{"1min":0.245,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.53400000000000003,"5min":0.65100000000000002,"15min":0.65100000000000002},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.40100000000000002,"15min":0.39500000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.34399999999999997}]},{"osd":3,"last update":"Fri Jun 14 22:39:34 2024","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.438,"15min":0.45100000000000001},"min":{"1min":0.27800000000000002,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.56799999999999995,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.38100000000000001,"5min":0.435,"15min":0.46000000000000002},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.5,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.374}]},{"osd":4,"last update":"Fri Jun 14 22:38:48 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.60499999999999998,"15min":0.60699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.70599999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58099999999999996,"15min":0.57899999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.72599999999999998,"5min":0.876,"15min":0.876},"last":0.55800000000000005}]},{"osd":5,"last update":"Fri Jun 14 22:39:13 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.58699999999999997,"15min":0.60499999999999998},"min":{"1min":0.38,"5min":0.312,"15min":0.312},"max":{"1min":0.65900000000000003,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.60999999999999999,"15min":0.61699999999999999},"min":{"1min":0.41099999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.77300000000000002,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.502}]},{"osd":6,"last update":"Fri Jun 14 22:39:37 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.64000000000000001,"15min":0.66400000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.90200000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.63,"15min":0.64600000000000002},"min":{"1min":0.45200000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.749,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.52600000000000002}]},{"osd":7,"last update":"Fri Jun 14 22:39:00 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.41499999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.77400000000000002,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.626,"15min":0.63200000000000001},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.77000000000000002,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.54900000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298064,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447088,"statfs":{"total":95995035648,"available":95689818112,"internally_reserved":0,"allocated":724992,"data_stored":316553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jun 14 22:38:38 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.34799999999999998,"15min":0.33300000000000002},"min":{"1min":0.214,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.52600000000000002,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.373,"15min":0.34799999999999998},"min":{"1min":0.23100000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.58399999999999996,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.48799999999999999}]},{"osd":2,"last update":"Fri Jun 14 22:39:06 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.42099999999999999,"15min":0.41699999999999998},"min":{"1min":0.26800000000000002,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.57199999999999995,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.373,"5min":0.38600000000000001,"15min":0.38900000000000001},"min":{"1min":0.252,"5min":0.246,"15min":0.246},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.496}]},{"osd":3,"last update":"Fri Jun 14 22:39:31 2024","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.40500000000000003,"15min":0.40899999999999997},"min":{"1min":0.247,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.61899999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.433,"5min":0.45600000000000002,"15min":0.46100000000000002},"min":{"1min":0.30399999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.627,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.51800000000000002}]},{"osd":4,"last update":"Fri Jun 14 22:38:53 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.56999999999999995},"min":{"1min":0.40300000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.72999999999999998,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.746},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.42699999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.748,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.95999999999999996}]},{"osd":5,"last update":"Fri Jun 14 22:39:17 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.60099999999999998,"15min":0.61099999999999999},"min":{"1min":0.41999999999999998,"5min":0.377,"15min":0.377},"max":{"1min":0.78200000000000003,"5min":1.032,"15min":1.032},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58099999999999996,"15min":0.59099999999999997},"min":{"1min":0.375,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91600000000000004,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.85499999999999998}]},{"osd":6,"last update":"Fri Jun 14 22:39:36 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.41199999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82399999999999995,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.60199999999999998,"15min":0.60099999999999998},"min":{"1min":0.47799999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.80800000000000005,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.94299999999999995}]},{"osd":7,"last update":"Fri Jun 14 22:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59599999999999997,"15min":0.59799999999999998},"min":{"1min":0.39400000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.63800000000000001,"15min":0.64400000000000002},"min":{"1min":0.50900000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72899999999999998,"5min":1.056,"15min":1.056},"last":0.92300000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738438,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298644,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446508,"statfs":{"total":95995035648,"available":95689224192,"internally_reserved":0,"allocated":1318912,"data_stored":906983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jun 14 22:38:41 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.36499999999999999,"15min":0.317},"min":{"1min":0.24099999999999999,"5min":0.214,"15min":0.214},"max":{"1min":1.4850000000000001,"5min":1.4850000000000001,"15min":1.4850000000000001},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.372,"15min":0.33100000000000002},"min":{"1min":0.28399999999999997,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":1.7,"5min":1.7,"15min":1.7},"last":0.46500000000000002}]},{"osd":2,"last update":"Fri Jun 14 22:39:03 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.40500000000000003,"15min":0.376},"min":{"1min":0.27000000000000002,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.58799999999999997,"5min":1.7230000000000001,"15min":1.7230000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.39800000000000002,"15min":0.36599999999999999},"min":{"1min":0.26900000000000002,"5min":0.248,"15min":0.248},"max":{"1min":0.91200000000000003,"5min":1.51,"15min":1.51},"last":0.40799999999999997}]},{"osd":3,"last update":"Fri Jun 14 22:39:29 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.40200000000000002,"15min":0.41599999999999998},"min":{"1min":0.25,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64800000000000002,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.46500000000000002,"15min":0.47899999999999998},"min":{"1min":0.32300000000000001,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.57299999999999995,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.49199999999999999}]},{"osd":4,"last update":"Fri Jun 14 22:38:46 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61099999999999999,"15min":0.61199999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":1.2},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.58399999999999996,"15min":0.58599999999999997},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":1.093}]},{"osd":5,"last update":"Fri Jun 14 22:39:14 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57099999999999995,"15min":0.57999999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.73299999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.60999999999999999,"15min":0.60699999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.88700000000000001,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Jun 14 22:39:40 2024","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.498,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90700000000000003,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60899999999999999,"15min":0.60199999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0289999999999999,"5min":1.609,"15min":1.609},"last":1.0289999999999999}]},{"osd":7,"last update":"Fri Jun 14 22:38:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.64500000000000002,"15min":0.65500000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.84099999999999997,"5min":1.673,"15min":1.673},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.59199999999999997,"15min":0.59299999999999997},"min":{"1min":0.45800000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.745,"5min":0.874,"15min":0.874},"last":0.68600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-14T22:39:43.346 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-14T22:39:43.347 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-14T22:39:43.347 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-14T22:39:43.347 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -- ceph health --format=json 2024-06-14T22:39:43.621 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:43.622 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:43 smithi046 ceph-mon[29586]: pgmap v79: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:43.622 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:43 smithi046 ceph-mon[38286]: pgmap v79: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:43.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:43 smithi178 ceph-mon[32942]: pgmap v79: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:44.615 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:44 smithi046 ceph-mon[29586]: from='client.24644 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-14T22:39:44.616 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:44 smithi046 ceph-mon[38286]: from='client.24644 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-14T22:39:44.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:44 smithi178 ceph-mon[32942]: from='client.24644 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-14T22:39:45.619 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-06-14T22:39:45.619 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-14T22:39:45.619 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:45 smithi046 ceph-mon[29586]: pgmap v80: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:45.619 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:45 smithi046 ceph-mon[29586]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:39:45.620 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:45 smithi046 ceph-mon[38286]: pgmap v80: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:45.620 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:45 smithi046 ceph-mon[38286]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:39:45.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:45 smithi178 ceph-mon[32942]: pgmap v80: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:45.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:45 smithi178 ceph-mon[32942]: from='mgr.14415 172.21.15.46:0/1634698668' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-06-14T22:39:46.331 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-14T22:39:46.332 INFO:teuthology.run_tasks:Running task print... 2024-06-14T22:39:46.342 INFO:teuthology.task.print:**** upgrading first half of cluster, with stress **** 2024-06-14T22:39:46.342 INFO:teuthology.run_tasks:Running task parallel... 2024-06-14T22:39:46.378 INFO:teuthology.task.parallel:starting parallel... 2024-06-14T22:39:46.379 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-06-14T22:39:46.379 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-06-14T22:39:46.384 INFO:tasks.workunit:Pulling workunits from ref quincy 2024-06-14T22:39:46.384 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-14T22:39:46.384 DEBUG:teuthology.orchestra.run.smithi046:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-14T22:39:46.385 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-06-14T22:39:46.385 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-06-14T22:39:46.385 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-06-14T22:39:46.385 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:quincy shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 -e sha1=6f4b09560ed399c3f139402a02f389c2fc8bb2cc -- bash -c 'ceph config set mgr mgr/cephadm/daemon_cache_timeout 60' 2024-06-14T22:39:46.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-14T22:39:46.403 INFO:teuthology.orchestra.run.smithi046.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-06-14T22:39:46.404 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-06-14T22:39:46.460 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-06-14T22:39:46.460 DEBUG:teuthology.orchestra.run.smithi046:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-06-14T22:39:46.516 INFO:tasks.workunit:timeout=3h 2024-06-14T22:39:46.516 INFO:tasks.workunit:cleanup=True 2024-06-14T22:39:46.516 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone --depth 1 --branch quincy https://github.com/cbodley/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 2024-06-14T22:39:46.572 INFO:tasks.workunit.client.0.smithi046.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-14T22:39:46.659 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/mon.a/config 2024-06-14T22:39:46.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:46 smithi046 ceph-mon[29586]: from='client.? 172.21.15.46:0/3925633275' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-14T22:39:46.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:46 smithi046 ceph-mon[38286]: from='client.? 172.21.15.46:0/3925633275' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-14T22:39:46.725 INFO:tasks.workunit.client.0.smithi046.stderr:warning: Could not find remote branch quincy to clone. 2024-06-14T22:39:46.725 INFO:tasks.workunit.client.0.smithi046.stderr:fatal: Remote branch quincy not found in upstream origin 2024-06-14T22:39:46.730 DEBUG:teuthology.orchestra.run:got remote process result: 128 2024-06-14T22:39:46.731 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/task/parallel.py", line 48, in task with parallel.parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/task/parallel.py", line 64, in _run_spawned mgr = run_tasks.run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/task/sequential.py", line 47, in task mgr = run_tasks.run_one_task(taskname, ctx=ctx, config=confg) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_cbodley_ceph_906f73e382c3916fe5507b8227cc56e63a20794f/qa/tasks/workunit.py", line 126, in task with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_cbodley_ceph_906f73e382c3916fe5507b8227cc56e63a20794f/qa/tasks/workunit.py", line 345, in _run_tests remote.run(logger=log.getChild(role), File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi046 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone --depth 1 --branch quincy https://github.com/cbodley/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0' 2024-06-14T22:39:46.965 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=602b20f258fd42c79b1ea2ff8e2a8c60 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/task/parallel.py", line 48, in task with parallel.parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/task/parallel.py", line 64, in _run_spawned mgr = run_tasks.run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/task/sequential.py", line 47, in task mgr = run_tasks.run_one_task(taskname, ctx=ctx, config=confg) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_cbodley_ceph_906f73e382c3916fe5507b8227cc56e63a20794f/qa/tasks/workunit.py", line 126, in task with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_cbodley_ceph_906f73e382c3916fe5507b8227cc56e63a20794f/qa/tasks/workunit.py", line 345, in _run_tests remote.run(logger=log.getChild(role), File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi046 with status 128: 'rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone --depth 1 --branch quincy https://github.com/cbodley/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0' 2024-06-14T22:39:46.968 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-14T22:39:46.981 INFO:tasks.cephadm:Teardown begin 2024-06-14T22:39:46.981 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-14T22:39:46.986 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:46 smithi178 ceph-mon[32942]: from='client.? 172.21.15.46:0/3925633275' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-14T22:39:47.012 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-14T22:39:47.047 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-14T22:39:47.048 DEBUG:teuthology.orchestra.run.smithi046:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-14T22:39:47.069 DEBUG:teuthology.orchestra.run.smithi178:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-14T22:39:47.106 INFO:tasks.cephadm:Stopping all daemons... 2024-06-14T22:39:47.107 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-14T22:39:47.107 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mon.a 2024-06-14T22:39:47.346 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:47 smithi046 systemd[1]: Stopping Ceph mon.a for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:39:47.693 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:47 smithi046 ceph-mon[29586]: pgmap v81: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:47.693 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:47 smithi046 ceph-mon[38286]: pgmap v81: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:47.775 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:47 smithi178 ceph-mon[32942]: pgmap v81: 73 pgs: 73 active+clean; 579 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-06-14T22:39:47.775 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:39:47 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x[34995]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:39:47] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-06-14T22:39:47.960 INFO:teuthology.orchestra.run.smithi046.stderr:Error: statfs /etc/ceph/ceph.client.admin.keyring: no such file or directory 2024-06-14T22:39:47.988 DEBUG:teuthology.orchestra.run:got remote process result: 125 2024-06-14T22:39:48.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:48 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-a[29563]: 2024-06-14T22:39:48.105+0000 7f95b62e1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-14T22:39:48.193 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:48 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-a[29563]: 2024-06-14T22:39:48.105+0000 7f95b62e1640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-14T22:39:48.193 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:48 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:39:48] ENGINE Bus STOPPING 2024-06-14T22:39:48.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:48 smithi046 podman[82111]: 2024-06-14 22:39:48.21759947 +0000 UTC m=+0.243494365 container died 4ab3984515da7ca2360c97fdd9f950c9bb2ad086792cff640dc8fb9b6204b5b3 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=quincy-e47ff4b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240612) 2024-06-14T22:39:48.492 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:48 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:39:48] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-14T22:39:48.492 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:48 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:39:48] ENGINE Bus STOPPED 2024-06-14T22:39:48.492 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:48 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:39:48] ENGINE Bus STARTING 2024-06-14T22:39:48.925 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:48 smithi046 podman[82111]: 2024-06-14 22:39:48.686141853 +0000 UTC m=+0.712036746 container cleanup 4ab3984515da7ca2360c97fdd9f950c9bb2ad086792cff640dc8fb9b6204b5b3 (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240612, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, RELEASE=quincy-e47ff4b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0) 2024-06-14T22:39:48.925 INFO:journalctl@ceph.mon.a.smithi046.stdout:Jun 14 22:39:48 smithi046 bash[82111]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-a 2024-06-14T22:39:48.925 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:48 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:39:48] ENGINE Serving on http://:::9283 2024-06-14T22:39:48.926 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:48 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:39:48] ENGINE Bus STARTED 2024-06-14T22:39:49.070 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mon.a.service' 2024-06-14T22:39:49.128 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-14T22:39:49.128 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-14T22:39:49.128 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-06-14T22:39:49.129 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mon.c 2024-06-14T22:39:49.193 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:49 smithi046 systemd[1]: Stopping Ceph mon.c for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:39:49.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:49 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-c[38263]: 2024-06-14T22:39:49.452+0000 7f9c92313640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-14T22:39:49.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:49 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-c[38263]: 2024-06-14T22:39:49.452+0000 7f9c92313640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-06-14T22:39:49.771 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:49 smithi046 podman[82352]: 2024-06-14 22:39:49.483250038 +0000 UTC m=+0.165293963 container died 2306f93e8c1b6d02a00989a576e5067b77bc6311ca0ce7babcd460d78debab25 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-c, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240612, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=quincy-e47ff4b, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-14T22:39:49.771 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:49 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:39:49] ENGINE Bus STOPPING 2024-06-14T22:39:49.772 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:49 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:39:49] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-06-14T22:39:49.772 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:49 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:39:49] ENGINE Bus STOPPED 2024-06-14T22:39:49.772 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:49 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:39:49] ENGINE Bus STARTING 2024-06-14T22:39:49.772 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:49 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:39:49] ENGINE Serving on http://:::9283 2024-06-14T22:39:49.772 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:49 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: [14/Jun/2024:22:39:49] ENGINE Bus STARTED 2024-06-14T22:39:50.113 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:49 smithi046 podman[82352]: 2024-06-14 22:39:49.873977016 +0000 UTC m=+0.556020980 container cleanup 2306f93e8c1b6d02a00989a576e5067b77bc6311ca0ce7babcd460d78debab25 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240612, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, RELEASE=quincy-e47ff4b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-06-14T22:39:50.113 INFO:journalctl@ceph.mon.c.smithi046.stdout:Jun 14 22:39:49 smithi046 bash[82352]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-c 2024-06-14T22:39:50.261 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mon.c.service' 2024-06-14T22:39:50.297 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-14T22:39:50.297 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-06-14T22:39:50.297 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-06-14T22:39:50.297 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mon.b 2024-06-14T22:39:50.624 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:50 smithi178 systemd[1]: Stopping Ceph mon.b for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:39:51.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:50 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-b[32919]: 2024-06-14T22:39:50.627+0000 7fc4f7de3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-14T22:39:51.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:50 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-b[32919]: 2024-06-14T22:39:50.627+0000 7fc4f7de3640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-06-14T22:39:51.025 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:50 smithi178 podman[67427]: 2024-06-14 22:39:50.771153009 +0000 UTC m=+0.255088703 container died cf042b827361861b30df3fec20ea522876b01fd7706b3f7846734dd486352577 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-b, org.label-schema.build-date=20240612, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=quincy-e47ff4b, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-14T22:39:51.389 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:51 smithi178 podman[67427]: 2024-06-14 22:39:51.150457785 +0000 UTC m=+0.634393476 container cleanup cf042b827361861b30df3fec20ea522876b01fd7706b3f7846734dd486352577 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240612, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, RELEASE=quincy-e47ff4b, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-14T22:39:51.389 INFO:journalctl@ceph.mon.b.smithi178.stdout:Jun 14 22:39:51 smithi178 bash[67427]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mon-b 2024-06-14T22:39:51.524 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mon.b.service' 2024-06-14T22:39:51.573 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-14T22:39:51.573 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-06-14T22:39:51.573 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-14T22:39:51.573 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mgr.y 2024-06-14T22:39:51.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:51 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: ::ffff:172.21.15.178 - - [14/Jun/2024:22:39:51] "GET /metrics HTTP/1.1" 200 239145 "" "Prometheus/2.43.0" 2024-06-14T22:39:51.693 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:51 smithi046 systemd[1]: Stopping Ceph mgr.y for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:39:52.193 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:51 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:39:51.897+0000 7f8d0c816640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-14T22:39:52.194 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:51 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y[29843]: 2024-06-14T22:39:51.897+0000 7f8d0c816640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-06-14T22:39:52.194 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:51 smithi046 podman[82493]: 2024-06-14 22:39:51.92967797 +0000 UTC m=+0.151994485 container died 47a0051cfde6c978a244f73690a092efae8641952d5ea343922c34e842f4bf5a (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240612, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=quincy-e47ff4b, io.buildah.version=1.36.0) 2024-06-14T22:39:52.668 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:52 smithi046 podman[82493]: 2024-06-14 22:39:52.386957189 +0000 UTC m=+0.609273738 container cleanup 47a0051cfde6c978a244f73690a092efae8641952d5ea343922c34e842f4bf5a (image=quay.ceph.io/ceph-ci/ceph:quincy, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240612, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=quincy-e47ff4b, GIT_BRANCH=HEAD) 2024-06-14T22:39:52.668 INFO:journalctl@ceph.mgr.y.smithi046.stdout:Jun 14 22:39:52 smithi046 bash[82493]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-y 2024-06-14T22:39:52.796 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mgr.y.service' 2024-06-14T22:39:52.833 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-14T22:39:52.833 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-14T22:39:52.833 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-14T22:39:52.833 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mgr.x 2024-06-14T22:39:53.158 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:39:52 smithi178 systemd[1]: Stopping Ceph mgr.x for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:39:53.418 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:39:53 smithi178 podman[67560]: 2024-06-14 22:39:53.162185296 +0000 UTC m=+0.127252428 container died c1f205eb48ebbd5ba3e5787678cdab417bfc3bec61c39bceecfc0fdaa2358902 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x, io.buildah.version=1.36.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-e47ff4b, ceph=True) 2024-06-14T22:39:53.775 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:39:53 smithi178 podman[67560]: 2024-06-14 22:39:53.487774668 +0000 UTC m=+0.452841794 container cleanup c1f205eb48ebbd5ba3e5787678cdab417bfc3bec61c39bceecfc0fdaa2358902 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=quincy-e47ff4b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240612) 2024-06-14T22:39:53.775 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:39:53 smithi178 bash[67560]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x 2024-06-14T22:39:53.775 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:39:53 smithi178 podman[67579]: 2024-06-14 22:39:53.639047777 +0000 UTC m=+0.473404446 container remove c1f205eb48ebbd5ba3e5787678cdab417bfc3bec61c39bceecfc0fdaa2358902 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-mgr-x, org.label-schema.build-date=20240612, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-e47ff4b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-06-14T22:39:53.776 INFO:journalctl@ceph.mgr.x.smithi178.stdout:Jun 14 22:39:53 smithi178 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-06-14T22:39:53.854 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@mgr.x.service' 2024-06-14T22:39:53.888 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-14T22:39:53.889 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-14T22:39:53.889 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-14T22:39:53.889 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.0 2024-06-14T22:39:54.193 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:39:53 smithi046 systemd[1]: Stopping Ceph osd.0 for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:39:54.683 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:39:54 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-0[42679]: 2024-06-14T22:39:54.202+0000 7fc7051c1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-14T22:39:54.683 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:39:54 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-0[42679]: 2024-06-14T22:39:54.202+0000 7fc7051c1640 -1 osd.0 71 *** Got signal Terminated *** 2024-06-14T22:39:54.684 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:39:54 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-0[42679]: 2024-06-14T22:39:54.202+0000 7fc7051c1640 -1 osd.0 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-14T22:39:59.871 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:39:59 smithi046 podman[82623]: 2024-06-14 22:39:59.608300931 +0000 UTC m=+5.504346785 container died 466381347f52aa198860570ac61bb58bad703ec2fc3221726cc01703e9e84721 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-0, org.label-schema.build-date=20240612, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , RELEASE=quincy-e47ff4b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-14T22:40:00.193 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:40:00 smithi046 podman[82623]: 2024-06-14 22:40:00.04961486 +0000 UTC m=+5.945660733 container cleanup 466381347f52aa198860570ac61bb58bad703ec2fc3221726cc01703e9e84721 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240612, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, RELEASE=quincy-e47ff4b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-06-14T22:40:00.193 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:40:00 smithi046 bash[82623]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-0 2024-06-14T22:40:00.693 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:40:00 smithi046 podman[82644]: 2024-06-14 22:40:00.384973461 +0000 UTC m=+0.770751723 container remove 466381347f52aa198860570ac61bb58bad703ec2fc3221726cc01703e9e84721 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240612, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, RELEASE=quincy-e47ff4b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-06-14T22:40:01.059 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:40:00 smithi046 podman[82741]: 2024-06-14 22:40:00.810748757 +0000 UTC m=+0.097769563 container create 39c16218d190f948c501d9037d0804fa24d55365cff03e9659349bce09a312e7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-0-deactivate, RELEASE=quincy-e47ff4b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240612, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-14T22:40:01.059 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:40:00 smithi046 podman[82741]: 2024-06-14 22:40:00.731095605 +0000 UTC m=+0.018116422 image pull 37f9eedc23c32ee6d04590e8a81233336d2c14cd278f52a2a2746a97e6ef2d95 quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580 2024-06-14T22:40:01.059 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:40:01 smithi046 podman[82741]: 2024-06-14 22:40:01.05669931 +0000 UTC m=+0.343720129 container init 39c16218d190f948c501d9037d0804fa24d55365cff03e9659349bce09a312e7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=quincy-e47ff4b, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0) 2024-06-14T22:40:01.059 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:40:01 smithi046 podman[82741]: 2024-06-14 22:40:01.062912646 +0000 UTC m=+0.349933461 container start 39c16218d190f948c501d9037d0804fa24d55365cff03e9659349bce09a312e7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-0-deactivate, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240612, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=quincy-e47ff4b, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-14T22:40:01.316 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:40:01 smithi046 podman[82741]: 2024-06-14 22:40:01.1650365 +0000 UTC m=+0.452057324 container attach 39c16218d190f948c501d9037d0804fa24d55365cff03e9659349bce09a312e7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, RELEASE=quincy-e47ff4b, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-06-14T22:40:01.613 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:40:01 smithi046 podman[82741]: 2024-06-14 22:40:01.319951066 +0000 UTC m=+0.606971874 container died 39c16218d190f948c501d9037d0804fa24d55365cff03e9659349bce09a312e7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=quincy-e47ff4b, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-14T22:40:01.870 INFO:journalctl@ceph.osd.0.smithi046.stdout:Jun 14 22:40:01 smithi046 podman[82784]: 2024-06-14 22:40:01.851752799 +0000 UTC m=+0.525905834 container remove 39c16218d190f948c501d9037d0804fa24d55365cff03e9659349bce09a312e7 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240612, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=quincy-e47ff4b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-14T22:40:01.876 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.0.service' 2024-06-14T22:40:01.917 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-14T22:40:01.917 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-14T22:40:01.917 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-14T22:40:01.917 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.1 2024-06-14T22:40:02.193 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:40:02 smithi046 systemd[1]: Stopping Ceph osd.1 for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:40:02.693 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:40:02 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-1[48593]: 2024-06-14T22:40:02.283+0000 7f4b5fb5d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-14T22:40:02.693 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:40:02 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-1[48593]: 2024-06-14T22:40:02.283+0000 7f4b5fb5d640 -1 osd.1 71 *** Got signal Terminated *** 2024-06-14T22:40:02.693 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:40:02 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-1[48593]: 2024-06-14T22:40:02.283+0000 7f4b5fb5d640 -1 osd.1 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-14T22:40:08.035 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:40:07 smithi046 podman[82870]: 2024-06-14 22:40:07.778396815 +0000 UTC m=+5.625862516 container died 9683fa437a6a9596b9a0dca560b067ad6f5f21f2a0b948fb9495a7e0b4752c00 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-1, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, RELEASE=quincy-e47ff4b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240612, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-14T22:40:08.369 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:40:08 smithi046 podman[82870]: 2024-06-14 22:40:08.221649181 +0000 UTC m=+6.069114882 container cleanup 9683fa437a6a9596b9a0dca560b067ad6f5f21f2a0b948fb9495a7e0b4752c00 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-1, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240612, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-e47ff4b, GIT_BRANCH=HEAD) 2024-06-14T22:40:08.369 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:40:08 smithi046 bash[82870]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-1 2024-06-14T22:40:08.693 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:40:08 smithi046 podman[82891]: 2024-06-14 22:40:08.372905553 +0000 UTC m=+0.587253183 container remove 9683fa437a6a9596b9a0dca560b067ad6f5f21f2a0b948fb9495a7e0b4752c00 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-1, ceph=True, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=quincy-e47ff4b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-14T22:40:09.071 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:40:08 smithi046 podman[82988]: 2024-06-14 22:40:08.790296197 +0000 UTC m=+0.114165244 container create cdedee4c1b878c8934f10fc4b1c23b9aec63f4d614f5488b88d05cf5325a716e (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240612, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=quincy-e47ff4b) 2024-06-14T22:40:09.072 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:40:08 smithi046 podman[82988]: 2024-06-14 22:40:08.694087635 +0000 UTC m=+0.017956707 image pull 37f9eedc23c32ee6d04590e8a81233336d2c14cd278f52a2a2746a97e6ef2d95 quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580 2024-06-14T22:40:09.072 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:40:08 smithi046 podman[82988]: 2024-06-14 22:40:08.992717105 +0000 UTC m=+0.316586160 container init cdedee4c1b878c8934f10fc4b1c23b9aec63f4d614f5488b88d05cf5325a716e (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240612, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, RELEASE=quincy-e47ff4b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-14T22:40:09.072 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:40:08 smithi046 podman[82988]: 2024-06-14 22:40:08.99896267 +0000 UTC m=+0.322831716 container start cdedee4c1b878c8934f10fc4b1c23b9aec63f4d614f5488b88d05cf5325a716e (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-1-deactivate, io.buildah.version=1.36.0, RELEASE=quincy-e47ff4b, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240612, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-14T22:40:09.072 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:40:09 smithi046 podman[82988]: 2024-06-14 22:40:09.075837651 +0000 UTC m=+0.399706708 container attach cdedee4c1b878c8934f10fc4b1c23b9aec63f4d614f5488b88d05cf5325a716e (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-e47ff4b, org.label-schema.build-date=20240612) 2024-06-14T22:40:09.443 INFO:journalctl@ceph.osd.1.smithi046.stdout:Jun 14 22:40:09 smithi046 podman[82988]: 2024-06-14 22:40:09.250812791 +0000 UTC m=+0.574681840 container died cdedee4c1b878c8934f10fc4b1c23b9aec63f4d614f5488b88d05cf5325a716e (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240612, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=quincy-e47ff4b, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-14T22:40:09.894 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.1.service' 2024-06-14T22:40:09.934 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-14T22:40:09.935 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-14T22:40:09.935 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-14T22:40:09.935 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.2 2024-06-14T22:40:10.193 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:40:10 smithi046 systemd[1]: Stopping Ceph osd.2 for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:40:10.693 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:40:10 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-2[54842]: 2024-06-14T22:40:10.301+0000 7f3c5bfbe640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-14T22:40:10.693 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:40:10 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-2[54842]: 2024-06-14T22:40:10.301+0000 7f3c5bfbe640 -1 osd.2 71 *** Got signal Terminated *** 2024-06-14T22:40:10.693 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:40:10 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-2[54842]: 2024-06-14T22:40:10.301+0000 7f3c5bfbe640 -1 osd.2 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-14T22:40:16.062 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:40:15 smithi046 podman[83115]: 2024-06-14 22:40:15.795013728 +0000 UTC m=+5.630963339 container died 041d4d3258bf111e38c78db9363345ac4316be50c29567ff203b38e725853c1e (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, RELEASE=quincy-e47ff4b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240612) 2024-06-14T22:40:16.443 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:40:16 smithi046 podman[83115]: 2024-06-14 22:40:16.257009484 +0000 UTC m=+6.092959099 container cleanup 041d4d3258bf111e38c78db9363345ac4316be50c29567ff203b38e725853c1e (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-2, org.label-schema.build-date=20240612, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=quincy-e47ff4b) 2024-06-14T22:40:16.443 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:40:16 smithi046 bash[83115]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-2 2024-06-14T22:40:16.889 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:40:16 smithi046 podman[83137]: 2024-06-14 22:40:16.53382584 +0000 UTC m=+0.737018077 container remove 041d4d3258bf111e38c78db9363345ac4316be50c29567ff203b38e725853c1e (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, RELEASE=quincy-e47ff4b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-14T22:40:17.182 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:40:16 smithi046 podman[83234]: 2024-06-14 22:40:16.892873193 +0000 UTC m=+0.110638519 container create 4b4844f83ca0bc667366a0655b7c538d704f272d02933c74b88e9df2d90826ec (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=quincy-e47ff4b, ceph=True) 2024-06-14T22:40:17.183 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:40:16 smithi046 podman[83234]: 2024-06-14 22:40:16.800309538 +0000 UTC m=+0.018074876 image pull 37f9eedc23c32ee6d04590e8a81233336d2c14cd278f52a2a2746a97e6ef2d95 quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580 2024-06-14T22:40:17.183 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:40:17 smithi046 podman[83234]: 2024-06-14 22:40:17.103729217 +0000 UTC m=+0.321494561 container init 4b4844f83ca0bc667366a0655b7c538d704f272d02933c74b88e9df2d90826ec (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-2-deactivate, RELEASE=quincy-e47ff4b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-14T22:40:17.184 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:40:17 smithi046 podman[83234]: 2024-06-14 22:40:17.109631671 +0000 UTC m=+0.327397008 container start 4b4844f83ca0bc667366a0655b7c538d704f272d02933c74b88e9df2d90826ec (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, RELEASE=quincy-e47ff4b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240612) 2024-06-14T22:40:17.184 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:40:17 smithi046 podman[83234]: 2024-06-14 22:40:17.186521933 +0000 UTC m=+0.404287285 container attach 4b4844f83ca0bc667366a0655b7c538d704f272d02933c74b88e9df2d90826ec (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, RELEASE=quincy-e47ff4b, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240612, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-14T22:40:17.443 INFO:journalctl@ceph.osd.2.smithi046.stdout:Jun 14 22:40:17 smithi046 podman[83234]: 2024-06-14 22:40:17.368460692 +0000 UTC m=+0.586226029 container died 4b4844f83ca0bc667366a0655b7c538d704f272d02933c74b88e9df2d90826ec (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, org.label-schema.license=GPLv2, RELEASE=quincy-e47ff4b, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240612) 2024-06-14T22:40:17.980 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.2.service' 2024-06-14T22:40:18.019 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-14T22:40:18.019 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-14T22:40:18.019 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-14T22:40:18.020 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.3 2024-06-14T22:40:18.290 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:40:18 smithi046 systemd[1]: Stopping Ceph osd.3 for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:40:18.693 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:40:18 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-3[61160]: 2024-06-14T22:40:18.421+0000 7f38571ed640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-14T22:40:18.693 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:40:18 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-3[61160]: 2024-06-14T22:40:18.421+0000 7f38571ed640 -1 osd.3 71 *** Got signal Terminated *** 2024-06-14T22:40:18.693 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:40:18 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-3[61160]: 2024-06-14T22:40:18.421+0000 7f38571ed640 -1 osd.3 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-14T22:40:21.275 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:20 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4[38354]: 2024-06-14T22:40:20.792+0000 7efef9265640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.334110+0000 front 2024-06-14T22:39:55.334046+0000 (oldest deadline 2024-06-14T22:40:20.634035+0000) 2024-06-14T22:40:22.025 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:21 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4[38354]: 2024-06-14T22:40:21.807+0000 7efef9265640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.334110+0000 front 2024-06-14T22:39:55.334046+0000 (oldest deadline 2024-06-14T22:40:20.634035+0000) 2024-06-14T22:40:22.026 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:21 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:21.596+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:22.586 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:22 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:22.285+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.491409+0000 front 2024-06-14T22:39:55.491471+0000 (oldest deadline 2024-06-14T22:40:21.391384+0000) 2024-06-14T22:40:23.025 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:22 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4[38354]: 2024-06-14T22:40:22.829+0000 7efef9265640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.334110+0000 front 2024-06-14T22:39:55.334046+0000 (oldest deadline 2024-06-14T22:40:20.634035+0000) 2024-06-14T22:40:23.026 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:22 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:22.589+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:23.443 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:40:23 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-3[61160]: 2024-06-14T22:40:23.022+0000 7f3852002640 -1 osd.3 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.845460+0000 front 2024-06-14T22:39:56.845550+0000 (oldest deadline 2024-06-14T22:40:22.745463+0000) 2024-06-14T22:40:23.775 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:23 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:23.298+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.491409+0000 front 2024-06-14T22:39:55.491471+0000 (oldest deadline 2024-06-14T22:40:21.391384+0000) 2024-06-14T22:40:23.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:23 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:23.545+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:23.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:23 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:23.471+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:24.074 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:40:23 smithi046 podman[83373]: 2024-06-14 22:40:23.796311566 +0000 UTC m=+5.523280327 container died cb30051337cf857cc468d3a47c779d7d2741248c549991b1eddc38b86db16235 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-3, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240612, RELEASE=quincy-e47ff4b, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-14T22:40:24.275 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:23 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4[38354]: 2024-06-14T22:40:23.829+0000 7efef9265640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.334110+0000 front 2024-06-14T22:39:55.334046+0000 (oldest deadline 2024-06-14T22:40:20.634035+0000) 2024-06-14T22:40:24.443 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:40:24 smithi046 podman[83373]: 2024-06-14 22:40:24.260137571 +0000 UTC m=+5.987106371 container cleanup cb30051337cf857cc468d3a47c779d7d2741248c549991b1eddc38b86db16235 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, RELEASE=quincy-e47ff4b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240612, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0) 2024-06-14T22:40:24.443 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:40:24 smithi046 bash[83373]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-3 2024-06-14T22:40:24.775 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:24.520+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:24.775 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:24.290+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.491409+0000 front 2024-06-14T22:39:55.491471+0000 (oldest deadline 2024-06-14T22:40:21.391384+0000) 2024-06-14T22:40:24.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:24.441+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:24.899 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:40:24 smithi046 podman[83397]: 2024-06-14 22:40:24.528032532 +0000 UTC m=+0.720994370 container remove cb30051337cf857cc468d3a47c779d7d2741248c549991b1eddc38b86db16235 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-3, RELEASE=quincy-e47ff4b, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240612) 2024-06-14T22:40:25.177 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:40:24 smithi046 podman[83493]: 2024-06-14 22:40:24.805454014 +0000 UTC m=+0.017439506 image pull 37f9eedc23c32ee6d04590e8a81233336d2c14cd278f52a2a2746a97e6ef2d95 quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580 2024-06-14T22:40:25.177 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:40:24 smithi046 podman[83493]: 2024-06-14 22:40:24.903589685 +0000 UTC m=+0.115575157 container create 00715ab74a885d754e2315725af6943d5bf1792353e43493f545945e7c6f0605 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-3-deactivate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240612, GIT_BRANCH=HEAD, RELEASE=quincy-e47ff4b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-14T22:40:25.177 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:40:25 smithi046 podman[83493]: 2024-06-14 22:40:25.097690195 +0000 UTC m=+0.309675682 container init 00715ab74a885d754e2315725af6943d5bf1792353e43493f545945e7c6f0605 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , RELEASE=quincy-e47ff4b, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.build-date=20240612, ceph=True) 2024-06-14T22:40:25.178 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:40:25 smithi046 podman[83493]: 2024-06-14 22:40:25.103299592 +0000 UTC m=+0.315285065 container start 00715ab74a885d754e2315725af6943d5bf1792353e43493f545945e7c6f0605 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240612, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=quincy-e47ff4b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True) 2024-06-14T22:40:25.178 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:40:25 smithi046 podman[83493]: 2024-06-14 22:40:25.180590067 +0000 UTC m=+0.392575592 container attach 00715ab74a885d754e2315725af6943d5bf1792353e43493f545945e7c6f0605 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, org.label-schema.build-date=20240612, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=quincy-e47ff4b, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47) 2024-06-14T22:40:25.241 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:24 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4[38354]: 2024-06-14T22:40:24.795+0000 7efef9265640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.334110+0000 front 2024-06-14T22:39:55.334046+0000 (oldest deadline 2024-06-14T22:40:20.634035+0000) 2024-06-14T22:40:25.443 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:40:25 smithi046 podman[83493]: 2024-06-14 22:40:25.357966095 +0000 UTC m=+0.569951580 container died 00715ab74a885d754e2315725af6943d5bf1792353e43493f545945e7c6f0605 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-3-deactivate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240612, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=quincy-e47ff4b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-14T22:40:25.519 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:25 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:25.244+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.491409+0000 front 2024-06-14T22:39:55.491471+0000 (oldest deadline 2024-06-14T22:40:21.391384+0000) 2024-06-14T22:40:25.520 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:25 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:25.470+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:25.775 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:25 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4[38354]: 2024-06-14T22:40:25.759+0000 7efef9265640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.334110+0000 front 2024-06-14T22:39:55.334046+0000 (oldest deadline 2024-06-14T22:40:20.634035+0000) 2024-06-14T22:40:25.775 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:25 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:25.523+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:25.881 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:40:25 smithi046 podman[83536]: 2024-06-14 22:40:25.867337658 +0000 UTC m=+0.502913468 container remove 00715ab74a885d754e2315725af6943d5bf1792353e43493f545945e7c6f0605 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-3-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=quincy-e47ff4b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0) 2024-06-14T22:40:25.882 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:40:25 smithi046 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.3.service: Deactivated successfully. 2024-06-14T22:40:25.882 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:40:25 smithi046 systemd[1]: Stopped Ceph osd.3 for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:40:25.882 INFO:journalctl@ceph.osd.3.smithi046.stdout:Jun 14 22:40:25 smithi046 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.3.service: Consumed 2.665s CPU time. 2024-06-14T22:40:25.883 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.3.service' 2024-06-14T22:40:25.924 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-14T22:40:25.924 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-14T22:40:25.924 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-14T22:40:25.925 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.4 2024-06-14T22:40:26.228 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:25 smithi178 systemd[1]: Stopping Ceph osd.4 for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:40:26.496 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:26 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:26.231+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.491409+0000 front 2024-06-14T22:39:55.491471+0000 (oldest deadline 2024-06-14T22:40:21.391384+0000) 2024-06-14T22:40:26.497 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:26 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:26.499+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:26.497 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:26 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:26.460+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:26.497 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:26 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4[38354]: 2024-06-14T22:40:26.232+0000 7efefb44a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-14T22:40:26.497 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:26 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4[38354]: 2024-06-14T22:40:26.232+0000 7efefb44a640 -1 osd.4 71 *** Got signal Terminated *** 2024-06-14T22:40:26.498 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:26 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4[38354]: 2024-06-14T22:40:26.232+0000 7efefb44a640 -1 osd.4 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-14T22:40:26.775 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:26 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4[38354]: 2024-06-14T22:40:26.710+0000 7efef9265640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.334110+0000 front 2024-06-14T22:39:55.334046+0000 (oldest deadline 2024-06-14T22:40:20.634035+0000) 2024-06-14T22:40:27.494 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:27 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:27.225+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.491409+0000 front 2024-06-14T22:39:55.491471+0000 (oldest deadline 2024-06-14T22:40:21.391384+0000) 2024-06-14T22:40:27.745 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:27 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:27.497+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:27.745 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:27 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:27.509+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:28.025 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:27 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4[38354]: 2024-06-14T22:40:27.748+0000 7efef9265640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.334110+0000 front 2024-06-14T22:39:55.334046+0000 (oldest deadline 2024-06-14T22:40:20.634035+0000) 2024-06-14T22:40:28.521 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:28 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:28.251+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.491409+0000 front 2024-06-14T22:39:55.491471+0000 (oldest deadline 2024-06-14T22:40:21.391384+0000) 2024-06-14T22:40:28.775 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:28 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4[38354]: 2024-06-14T22:40:28.734+0000 7efef9265640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.334110+0000 front 2024-06-14T22:39:55.334046+0000 (oldest deadline 2024-06-14T22:40:20.634035+0000) 2024-06-14T22:40:28.775 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:28 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:28.534+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:28.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:28 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:28.524+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:29.513 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:29 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:29.252+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.491409+0000 front 2024-06-14T22:39:55.491471+0000 (oldest deadline 2024-06-14T22:40:21.391384+0000) 2024-06-14T22:40:29.775 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:29 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4[38354]: 2024-06-14T22:40:29.734+0000 7efef9265640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.334110+0000 front 2024-06-14T22:39:55.334046+0000 (oldest deadline 2024-06-14T22:40:20.634035+0000) 2024-06-14T22:40:29.775 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:29 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:29.543+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:29.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:29 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:29.516+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:30.525 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:30 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:30.265+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.491409+0000 front 2024-06-14T22:39:55.491471+0000 (oldest deadline 2024-06-14T22:40:21.391384+0000) 2024-06-14T22:40:30.525 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:30 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:30.491+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:31.025 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:30 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:30.575+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:31.025 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:30 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4[38354]: 2024-06-14T22:40:30.759+0000 7efef9265640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.334110+0000 front 2024-06-14T22:39:55.334046+0000 (oldest deadline 2024-06-14T22:40:20.634035+0000) 2024-06-14T22:40:31.025 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:30 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4[38354]: 2024-06-14T22:40:30.759+0000 7efef9265640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:05.735634+0000 front 2024-06-14T22:40:05.735654+0000 (oldest deadline 2024-06-14T22:40:29.835437+0000) 2024-06-14T22:40:31.555 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:31 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:31.279+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.491409+0000 front 2024-06-14T22:39:55.491471+0000 (oldest deadline 2024-06-14T22:40:21.391384+0000) 2024-06-14T22:40:31.555 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:31 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:31.497+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:31.555 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:31 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:31.497+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:32.002 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:31 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:31.558+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:32.002 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:31 smithi178 podman[67707]: 2024-06-14 22:40:31.702067731 +0000 UTC m=+5.573218454 container died 784c939a0a342c12fa8fd4d19d737934ba937d05e37d7604d08cc6bba06a602c (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4, ceph=True, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240612, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=quincy-e47ff4b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-14T22:40:32.275 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:32 smithi178 podman[67707]: 2024-06-14 22:40:32.188034794 +0000 UTC m=+6.059185558 container cleanup 784c939a0a342c12fa8fd4d19d737934ba937d05e37d7604d08cc6bba06a602c (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=quincy-e47ff4b, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, org.label-schema.build-date=20240612, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-14T22:40:32.276 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:32 smithi178 bash[67707]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4 2024-06-14T22:40:32.771 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:32 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:32.520+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:32.772 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:32 smithi178 podman[67729]: 2024-06-14 22:40:32.440017092 +0000 UTC m=+0.728914788 container remove 784c939a0a342c12fa8fd4d19d737934ba937d05e37d7604d08cc6bba06a602c (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240612, io.buildah.version=1.36.0, RELEASE=quincy-e47ff4b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-06-14T22:40:32.772 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:32 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:32.314+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.491409+0000 front 2024-06-14T22:39:55.491471+0000 (oldest deadline 2024-06-14T22:40:21.391384+0000) 2024-06-14T22:40:32.772 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:32 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:32.502+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:32.772 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:32 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:32.502+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:33.025 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:32 smithi178 podman[67822]: 2024-06-14 22:40:32.678070347 +0000 UTC m=+0.016966162 image pull 37f9eedc23c32ee6d04590e8a81233336d2c14cd278f52a2a2746a97e6ef2d95 quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580 2024-06-14T22:40:33.025 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:32 smithi178 podman[67822]: 2024-06-14 22:40:32.790822487 +0000 UTC m=+0.129718292 container create 3b8817a56a8cf8a4190afbbe07e0066c165aaf49ce9ca4f9e979569fb0c778ea (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=quincy-e47ff4b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240612, ceph=True, org.label-schema.schema-version=1.0) 2024-06-14T22:40:33.026 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:33 smithi178 podman[67822]: 2024-06-14 22:40:33.018214136 +0000 UTC m=+0.357109947 container init 3b8817a56a8cf8a4190afbbe07e0066c165aaf49ce9ca4f9e979569fb0c778ea (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240612, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-e47ff4b, org.label-schema.schema-version=1.0, ceph=True) 2024-06-14T22:40:33.026 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:33 smithi178 podman[67822]: 2024-06-14 22:40:33.024052245 +0000 UTC m=+0.362948051 container start 3b8817a56a8cf8a4190afbbe07e0066c165aaf49ce9ca4f9e979569fb0c778ea (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=quincy-e47ff4b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=) 2024-06-14T22:40:33.458 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:33 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:33.342+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.491409+0000 front 2024-06-14T22:39:55.491471+0000 (oldest deadline 2024-06-14T22:40:21.391384+0000) 2024-06-14T22:40:33.458 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:33 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:33.342+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.592991+0000 front 2024-06-14T22:40:06.592898+0000 (oldest deadline 2024-06-14T22:40:32.492792+0000) 2024-06-14T22:40:33.458 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:33 smithi178 podman[67822]: 2024-06-14 22:40:33.126655628 +0000 UTC m=+0.465551434 container attach 3b8817a56a8cf8a4190afbbe07e0066c165aaf49ce9ca4f9e979569fb0c778ea (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240612, RELEASE=quincy-e47ff4b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-14T22:40:33.458 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:33 smithi178 podman[67822]: 2024-06-14 22:40:33.276659625 +0000 UTC m=+0.615555435 container died 3b8817a56a8cf8a4190afbbe07e0066c165aaf49ce9ca4f9e979569fb0c778ea (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240612, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=quincy-e47ff4b, maintainer=Guillaume Abrioux ) 2024-06-14T22:40:33.719 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:33 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:33.479+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:33.719 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:33 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:33.479+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:07.276539+0000 front 2024-06-14T22:40:07.276584+0000 (oldest deadline 2024-06-14T22:40:33.176397+0000) 2024-06-14T22:40:33.719 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:33 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:33.461+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:33.719 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:33 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:33.461+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:33.957 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.4.service' 2024-06-14T22:40:33.981 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:33 smithi178 podman[67865]: 2024-06-14 22:40:33.938772031 +0000 UTC m=+0.652831821 container remove 3b8817a56a8cf8a4190afbbe07e0066c165aaf49ce9ca4f9e979569fb0c778ea (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=quincy-e47ff4b, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.36.0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240612, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-14T22:40:33.981 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:33 smithi178 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.4.service: Deactivated successfully. 2024-06-14T22:40:33.981 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:33 smithi178 systemd[1]: Stopped Ceph osd.4 for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:40:33.981 INFO:journalctl@ceph.osd.4.smithi178.stdout:Jun 14 22:40:33 smithi178 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.4.service: Consumed 2.594s CPU time. 2024-06-14T22:40:33.997 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-14T22:40:33.997 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-14T22:40:33.997 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-14T22:40:33.997 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.5 2024-06-14T22:40:34.275 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:34 smithi178 systemd[1]: Stopping Ceph osd.5 for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:40:34.775 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:34 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:34.360+0000 7fdf83ab2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-14T22:40:34.776 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:34 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:34.360+0000 7fdf83ab2640 -1 osd.5 71 *** Got signal Terminated *** 2024-06-14T22:40:34.776 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:34 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:34.360+0000 7fdf83ab2640 -1 osd.5 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-14T22:40:34.776 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:34 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:34.389+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.491409+0000 front 2024-06-14T22:39:55.491471+0000 (oldest deadline 2024-06-14T22:40:21.391384+0000) 2024-06-14T22:40:34.776 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:34 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:34.389+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.592991+0000 front 2024-06-14T22:40:06.592898+0000 (oldest deadline 2024-06-14T22:40:32.492792+0000) 2024-06-14T22:40:34.777 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:34 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:34.525+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:34.777 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:34 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:34.525+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:07.276539+0000 front 2024-06-14T22:40:07.276584+0000 (oldest deadline 2024-06-14T22:40:33.176397+0000) 2024-06-14T22:40:34.777 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:34 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:34.460+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:34.777 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:34 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:34.460+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:35.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:35 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:35.411+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:35.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:35 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:35.411+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:35.776 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:35 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:35.384+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.491409+0000 front 2024-06-14T22:39:55.491471+0000 (oldest deadline 2024-06-14T22:40:21.391384+0000) 2024-06-14T22:40:35.776 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:35 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:35.384+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.592991+0000 front 2024-06-14T22:40:06.592898+0000 (oldest deadline 2024-06-14T22:40:32.492792+0000) 2024-06-14T22:40:35.777 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:35 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:35.494+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:35.777 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:35 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:35.494+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:07.276539+0000 front 2024-06-14T22:40:07.276584+0000 (oldest deadline 2024-06-14T22:40:33.176397+0000) 2024-06-14T22:40:36.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:36 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:36.415+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:36.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:36 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:36.415+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:36.776 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:36 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:36.355+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.491409+0000 front 2024-06-14T22:39:55.491471+0000 (oldest deadline 2024-06-14T22:40:21.391384+0000) 2024-06-14T22:40:36.776 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:36 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:36.355+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.592991+0000 front 2024-06-14T22:40:06.592898+0000 (oldest deadline 2024-06-14T22:40:32.492792+0000) 2024-06-14T22:40:36.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:36 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:36.480+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:36.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:36 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:36.480+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:07.276539+0000 front 2024-06-14T22:40:07.276584+0000 (oldest deadline 2024-06-14T22:40:33.176397+0000) 2024-06-14T22:40:36.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:36 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:36.480+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:13.176924+0000 front 2024-06-14T22:40:13.176929+0000 (oldest deadline 2024-06-14T22:40:36.076803+0000) 2024-06-14T22:40:37.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:37 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:37.388+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:37.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:37 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:37.388+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:37.776 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:37 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:37.388+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.491409+0000 front 2024-06-14T22:39:55.491471+0000 (oldest deadline 2024-06-14T22:40:21.391384+0000) 2024-06-14T22:40:37.776 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:37 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:37.388+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.592991+0000 front 2024-06-14T22:40:06.592898+0000 (oldest deadline 2024-06-14T22:40:32.492792+0000) 2024-06-14T22:40:37.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:37 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:37.471+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:37.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:37 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:37.471+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:07.276539+0000 front 2024-06-14T22:40:07.276584+0000 (oldest deadline 2024-06-14T22:40:33.176397+0000) 2024-06-14T22:40:37.777 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:37 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:37.471+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:13.176924+0000 front 2024-06-14T22:40:13.176929+0000 (oldest deadline 2024-06-14T22:40:36.076803+0000) 2024-06-14T22:40:38.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:38 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:38.382+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:38.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:38 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:38.382+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:38.776 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:38 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:38.374+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:55.491409+0000 front 2024-06-14T22:39:55.491471+0000 (oldest deadline 2024-06-14T22:40:21.391384+0000) 2024-06-14T22:40:38.776 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:38 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:38.374+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.592991+0000 front 2024-06-14T22:40:06.592898+0000 (oldest deadline 2024-06-14T22:40:32.492792+0000) 2024-06-14T22:40:38.776 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:38 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5[44245]: 2024-06-14T22:40:38.374+0000 7fdf800ca640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:12.493197+0000 front 2024-06-14T22:40:12.493247+0000 (oldest deadline 2024-06-14T22:40:37.793232+0000) 2024-06-14T22:40:38.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:38 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:38.516+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:38.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:38 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:38.516+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:07.276539+0000 front 2024-06-14T22:40:07.276584+0000 (oldest deadline 2024-06-14T22:40:33.176397+0000) 2024-06-14T22:40:38.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:38 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:38.516+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:13.176924+0000 front 2024-06-14T22:40:13.176929+0000 (oldest deadline 2024-06-14T22:40:36.076803+0000) 2024-06-14T22:40:39.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:39 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:39.335+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:39.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:39 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:39.335+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:39.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:39 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:39.540+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:39.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:39 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:39.540+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:07.276539+0000 front 2024-06-14T22:40:07.276584+0000 (oldest deadline 2024-06-14T22:40:33.176397+0000) 2024-06-14T22:40:39.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:39 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:39.540+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:13.176924+0000 front 2024-06-14T22:40:13.176929+0000 (oldest deadline 2024-06-14T22:40:36.076803+0000) 2024-06-14T22:40:40.197 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:39 smithi178 podman[67960]: 2024-06-14 22:40:39.906629851 +0000 UTC m=+5.655839180 container died 6d83af630912d3adacf1ca2395a5efb0e1d9d0e8d5557f7e69d6885e15f7205a (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=quincy-e47ff4b, io.buildah.version=1.36.0, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240612, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-14T22:40:40.525 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:40 smithi178 podman[67960]: 2024-06-14 22:40:40.357942969 +0000 UTC m=+6.107152343 container cleanup 6d83af630912d3adacf1ca2395a5efb0e1d9d0e8d5557f7e69d6885e15f7205a (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.36.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=quincy-e47ff4b, org.label-schema.build-date=20240612) 2024-06-14T22:40:40.526 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:40 smithi178 bash[67960]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5 2024-06-14T22:40:40.526 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:40 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:40.360+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:40.526 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:40 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:40.360+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:40.950 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:40 smithi178 podman[67984]: 2024-06-14 22:40:40.553190378 +0000 UTC m=+0.645878669 container remove 6d83af630912d3adacf1ca2395a5efb0e1d9d0e8d5557f7e69d6885e15f7205a (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240612, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True, io.buildah.version=1.36.0, RELEASE=quincy-e47ff4b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-06-14T22:40:40.950 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:40 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:40.529+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:40.950 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:40 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:40.529+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:07.276539+0000 front 2024-06-14T22:40:07.276584+0000 (oldest deadline 2024-06-14T22:40:33.176397+0000) 2024-06-14T22:40:40.950 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:40 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:40.529+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:13.176924+0000 front 2024-06-14T22:40:13.176929+0000 (oldest deadline 2024-06-14T22:40:36.076803+0000) 2024-06-14T22:40:41.242 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:40 smithi178 podman[68077]: 2024-06-14 22:40:40.953952534 +0000 UTC m=+0.090754194 container create 8959ff6f53126d30f14f97b8531c2a6a0521cfb4357fa4023d981f3e62ecbec6 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, RELEASE=quincy-e47ff4b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240612, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-14T22:40:41.242 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:40 smithi178 podman[68077]: 2024-06-14 22:40:40.880309885 +0000 UTC m=+0.017111542 image pull 37f9eedc23c32ee6d04590e8a81233336d2c14cd278f52a2a2746a97e6ef2d95 quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580 2024-06-14T22:40:41.242 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:41 smithi178 podman[68077]: 2024-06-14 22:40:41.14816753 +0000 UTC m=+0.284969187 container init 8959ff6f53126d30f14f97b8531c2a6a0521cfb4357fa4023d981f3e62ecbec6 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-e47ff4b, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240612, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-14T22:40:41.242 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:41 smithi178 podman[68077]: 2024-06-14 22:40:41.153640275 +0000 UTC m=+0.290441943 container start 8959ff6f53126d30f14f97b8531c2a6a0521cfb4357fa4023d981f3e62ecbec6 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240612, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=quincy-e47ff4b, org.label-schema.schema-version=1.0) 2024-06-14T22:40:41.242 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:41 smithi178 podman[68077]: 2024-06-14 22:40:41.245808977 +0000 UTC m=+0.382610635 container attach 8959ff6f53126d30f14f97b8531c2a6a0521cfb4357fa4023d981f3e62ecbec6 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5-deactivate, RELEASE=quincy-e47ff4b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240612, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, ceph=True) 2024-06-14T22:40:41.525 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:41.356+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:41.525 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:41.356+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:41.526 INFO:journalctl@ceph.osd.5.smithi178.stdout:Jun 14 22:40:41 smithi178 podman[68077]: 2024-06-14 22:40:41.401863486 +0000 UTC m=+0.538665146 container died 8959ff6f53126d30f14f97b8531c2a6a0521cfb4357fa4023d981f3e62ecbec6 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-5-deactivate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240612, RELEASE=quincy-e47ff4b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-14T22:40:42.005 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:41.554+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:42.006 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:41.554+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:07.276539+0000 front 2024-06-14T22:40:07.276584+0000 (oldest deadline 2024-06-14T22:40:33.176397+0000) 2024-06-14T22:40:42.006 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:41 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:41.554+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:13.176924+0000 front 2024-06-14T22:40:13.176929+0000 (oldest deadline 2024-06-14T22:40:36.076803+0000) 2024-06-14T22:40:42.030 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.5.service' 2024-06-14T22:40:42.071 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-14T22:40:42.072 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-14T22:40:42.072 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-14T22:40:42.072 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.6 2024-06-14T22:40:42.275 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:42 smithi178 systemd[1]: Stopping Ceph osd.6 for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:40:42.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:42 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:42.332+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:42.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:42 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:42.332+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:42.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:42 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:42.332+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:10.836533+0000 front 2024-06-14T22:40:10.836498+0000 (oldest deadline 2024-06-14T22:40:41.436814+0000) 2024-06-14T22:40:42.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:42 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:42.423+0000 7f130ba65640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-14T22:40:42.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:42 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:42.423+0000 7f130ba65640 -1 osd.6 71 *** Got signal Terminated *** 2024-06-14T22:40:42.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:42 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:42.423+0000 7f130ba65640 -1 osd.6 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-14T22:40:42.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:42 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:42.509+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:42.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:42 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:42.509+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:07.276539+0000 front 2024-06-14T22:40:07.276584+0000 (oldest deadline 2024-06-14T22:40:33.176397+0000) 2024-06-14T22:40:42.777 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:42 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:42.509+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:13.176924+0000 front 2024-06-14T22:40:13.176929+0000 (oldest deadline 2024-06-14T22:40:36.076803+0000) 2024-06-14T22:40:43.775 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:43 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:43.528+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:43.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:43 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:43.528+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:07.276539+0000 front 2024-06-14T22:40:07.276584+0000 (oldest deadline 2024-06-14T22:40:33.176397+0000) 2024-06-14T22:40:43.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:43 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:43.528+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:13.176924+0000 front 2024-06-14T22:40:13.176929+0000 (oldest deadline 2024-06-14T22:40:36.076803+0000) 2024-06-14T22:40:43.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:43 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:43.364+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:43.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:43 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:43.364+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:43.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:43 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:43.364+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:10.836533+0000 front 2024-06-14T22:40:10.836498+0000 (oldest deadline 2024-06-14T22:40:41.436814+0000) 2024-06-14T22:40:44.775 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:44.500+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:44.775 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:44.500+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:07.276539+0000 front 2024-06-14T22:40:07.276584+0000 (oldest deadline 2024-06-14T22:40:33.176397+0000) 2024-06-14T22:40:44.776 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:44.500+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:13.176924+0000 front 2024-06-14T22:40:13.176929+0000 (oldest deadline 2024-06-14T22:40:36.076803+0000) 2024-06-14T22:40:44.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:44.353+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:44.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:44.353+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:44.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:44 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:44.353+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:10.836533+0000 front 2024-06-14T22:40:10.836498+0000 (oldest deadline 2024-06-14T22:40:41.436814+0000) 2024-06-14T22:40:45.775 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:45 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:45.469+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:45.775 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:45 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:45.469+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:07.276539+0000 front 2024-06-14T22:40:07.276584+0000 (oldest deadline 2024-06-14T22:40:33.176397+0000) 2024-06-14T22:40:45.775 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:45 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:45.469+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:13.176924+0000 front 2024-06-14T22:40:13.176929+0000 (oldest deadline 2024-06-14T22:40:36.076803+0000) 2024-06-14T22:40:45.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:45 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:45.391+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:45.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:45 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:45.391+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:45.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:45 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:45.391+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:10.836533+0000 front 2024-06-14T22:40:10.836498+0000 (oldest deadline 2024-06-14T22:40:41.436814+0000) 2024-06-14T22:40:46.775 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:46 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:46.518+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:56.675484+0000 front 2024-06-14T22:39:56.675539+0000 (oldest deadline 2024-06-14T22:40:21.375507+0000) 2024-06-14T22:40:46.775 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:46 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:46.518+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:07.276539+0000 front 2024-06-14T22:40:07.276584+0000 (oldest deadline 2024-06-14T22:40:33.176397+0000) 2024-06-14T22:40:46.775 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:46 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6[50195]: 2024-06-14T22:40:46.518+0000 7f130687a640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:13.176924+0000 front 2024-06-14T22:40:13.176929+0000 (oldest deadline 2024-06-14T22:40:36.076803+0000) 2024-06-14T22:40:46.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:46 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:46.412+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:46.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:46 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:46.412+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:46.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:46 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:46.412+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:10.836533+0000 front 2024-06-14T22:40:10.836498+0000 (oldest deadline 2024-06-14T22:40:41.436814+0000) 2024-06-14T22:40:46.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:46 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:46.412+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6830 osd.3 since back 2024-06-14T22:40:21.437523+0000 front 2024-06-14T22:40:21.437580+0000 (oldest deadline 2024-06-14T22:40:45.537314+0000) 2024-06-14T22:40:47.748 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:47 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:47.364+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:47.749 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:47 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:47.364+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:47.749 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:47 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:47.364+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:10.836533+0000 front 2024-06-14T22:40:10.836498+0000 (oldest deadline 2024-06-14T22:40:41.436814+0000) 2024-06-14T22:40:47.749 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:47 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:47.364+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6830 osd.3 since back 2024-06-14T22:40:21.437523+0000 front 2024-06-14T22:40:21.437580+0000 (oldest deadline 2024-06-14T22:40:45.537314+0000) 2024-06-14T22:40:48.025 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:47 smithi178 podman[68203]: 2024-06-14 22:40:47.752278795 +0000 UTC m=+5.447725286 container died 482a679c9db79bd428ccffebb66436b4744e94aaf91f54af6d070ecaf18d20c4 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=quincy-e47ff4b, io.buildah.version=1.36.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612) 2024-06-14T22:40:48.349 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:48 smithi178 podman[68203]: 2024-06-14 22:40:48.212031743 +0000 UTC m=+5.907478235 container cleanup 482a679c9db79bd428ccffebb66436b4744e94aaf91f54af6d070ecaf18d20c4 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6, RELEASE=quincy-e47ff4b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.36.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240612, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-06-14T22:40:48.350 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:48 smithi178 bash[68203]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6 2024-06-14T22:40:48.775 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:48 smithi178 podman[68224]: 2024-06-14 22:40:48.497051096 +0000 UTC m=+0.736702535 container remove 482a679c9db79bd428ccffebb66436b4744e94aaf91f54af6d070ecaf18d20c4 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6, ceph=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, RELEASE=quincy-e47ff4b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240612, GIT_BRANCH=HEAD) 2024-06-14T22:40:48.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:48 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:48.352+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:48.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:48 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:48.352+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:48.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:48 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:48.352+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:10.836533+0000 front 2024-06-14T22:40:10.836498+0000 (oldest deadline 2024-06-14T22:40:41.436814+0000) 2024-06-14T22:40:48.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:48 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:48.352+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6830 osd.3 since back 2024-06-14T22:40:21.437523+0000 front 2024-06-14T22:40:21.437580+0000 (oldest deadline 2024-06-14T22:40:45.537314+0000) 2024-06-14T22:40:49.129 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:48 smithi178 podman[68322]: 2024-06-14 22:40:48.823010747 +0000 UTC m=+0.109153893 container create 3ab0a2320ae7fab3fef8f58e6558fb3dd4d889a31235a396fe1b6db02aa3ca78 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.36.0, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612, RELEASE=quincy-e47ff4b) 2024-06-14T22:40:49.129 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:48 smithi178 podman[68322]: 2024-06-14 22:40:48.730915375 +0000 UTC m=+0.017058521 image pull 37f9eedc23c32ee6d04590e8a81233336d2c14cd278f52a2a2746a97e6ef2d95 quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580 2024-06-14T22:40:49.129 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:49 smithi178 podman[68322]: 2024-06-14 22:40:49.04166439 +0000 UTC m=+0.327807540 container init 3ab0a2320ae7fab3fef8f58e6558fb3dd4d889a31235a396fe1b6db02aa3ca78 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6-deactivate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, GIT_CLEAN=True, RELEASE=quincy-e47ff4b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240612, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-14T22:40:49.129 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:49 smithi178 podman[68322]: 2024-06-14 22:40:49.047891857 +0000 UTC m=+0.334034994 container start 3ab0a2320ae7fab3fef8f58e6558fb3dd4d889a31235a396fe1b6db02aa3ca78 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=quincy-e47ff4b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240612, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.36.0) 2024-06-14T22:40:49.130 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:49 smithi178 podman[68322]: 2024-06-14 22:40:49.133117293 +0000 UTC m=+0.419260438 container attach 3ab0a2320ae7fab3fef8f58e6558fb3dd4d889a31235a396fe1b6db02aa3ca78 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240612, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=quincy-e47ff4b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-14T22:40:49.525 INFO:journalctl@ceph.osd.6.smithi178.stdout:Jun 14 22:40:49 smithi178 podman[68322]: 2024-06-14 22:40:49.296643679 +0000 UTC m=+0.582786823 container died 3ab0a2320ae7fab3fef8f58e6558fb3dd4d889a31235a396fe1b6db02aa3ca78 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.schema-version=1.0, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240612, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=quincy-e47ff4b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-14T22:40:49.525 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:49 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:49.325+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:49.526 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:49 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:49.325+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:49.526 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:49 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:49.325+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:10.836533+0000 front 2024-06-14T22:40:10.836498+0000 (oldest deadline 2024-06-14T22:40:41.436814+0000) 2024-06-14T22:40:49.526 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:49 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:49.325+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6830 osd.3 since back 2024-06-14T22:40:21.437523+0000 front 2024-06-14T22:40:21.437580+0000 (oldest deadline 2024-06-14T22:40:45.537314+0000) 2024-06-14T22:40:49.939 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.6.service' 2024-06-14T22:40:49.978 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-14T22:40:49.978 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-14T22:40:49.978 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-14T22:40:49.979 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.7 2024-06-14T22:40:50.275 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:50 smithi178 systemd[1]: Stopping Ceph osd.7 for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:40:50.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:50 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:50.291+0000 7f243599e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-14T22:40:50.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:50 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:50.291+0000 7f243599e640 -1 osd.7 71 *** Got signal Terminated *** 2024-06-14T22:40:50.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:50 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:50.291+0000 7f243599e640 -1 osd.7 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-14T22:40:50.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:50 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:50.349+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:50.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:50 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:50.349+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:50.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:50 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:50.349+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:10.836533+0000 front 2024-06-14T22:40:10.836498+0000 (oldest deadline 2024-06-14T22:40:41.436814+0000) 2024-06-14T22:40:50.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:50 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:50.349+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6830 osd.3 since back 2024-06-14T22:40:21.437523+0000 front 2024-06-14T22:40:21.437580+0000 (oldest deadline 2024-06-14T22:40:45.537314+0000) 2024-06-14T22:40:51.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:51 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:51.346+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:51.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:51 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:51.346+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:51.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:51 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:51.346+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:10.836533+0000 front 2024-06-14T22:40:10.836498+0000 (oldest deadline 2024-06-14T22:40:41.436814+0000) 2024-06-14T22:40:51.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:51 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:51.346+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6830 osd.3 since back 2024-06-14T22:40:21.437523+0000 front 2024-06-14T22:40:21.437580+0000 (oldest deadline 2024-06-14T22:40:45.537314+0000) 2024-06-14T22:40:52.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:52 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:52.307+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:52.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:52 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:52.307+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:52.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:52 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:52.307+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:10.836533+0000 front 2024-06-14T22:40:10.836498+0000 (oldest deadline 2024-06-14T22:40:41.436814+0000) 2024-06-14T22:40:52.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:52 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:52.307+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6830 osd.3 since back 2024-06-14T22:40:21.437523+0000 front 2024-06-14T22:40:21.437580+0000 (oldest deadline 2024-06-14T22:40:45.537314+0000) 2024-06-14T22:40:53.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:53 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:53.341+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:53.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:53 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:53.341+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:53.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:53 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:53.341+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:10.836533+0000 front 2024-06-14T22:40:10.836498+0000 (oldest deadline 2024-06-14T22:40:41.436814+0000) 2024-06-14T22:40:53.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:53 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:53.341+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6830 osd.3 since back 2024-06-14T22:40:21.437523+0000 front 2024-06-14T22:40:21.437580+0000 (oldest deadline 2024-06-14T22:40:45.537314+0000) 2024-06-14T22:40:54.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:54 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:54.388+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-06-14T22:39:58.034951+0000 front 2024-06-14T22:39:58.034865+0000 (oldest deadline 2024-06-14T22:40:22.734749+0000) 2024-06-14T22:40:54.775 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:54 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:54.388+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6814 osd.1 since back 2024-06-14T22:40:06.835645+0000 front 2024-06-14T22:40:06.835588+0000 (oldest deadline 2024-06-14T22:40:30.835864+0000) 2024-06-14T22:40:54.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:54 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:54.388+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6822 osd.2 since back 2024-06-14T22:40:10.836533+0000 front 2024-06-14T22:40:10.836498+0000 (oldest deadline 2024-06-14T22:40:41.436814+0000) 2024-06-14T22:40:54.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:54 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:54.388+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.46:6830 osd.3 since back 2024-06-14T22:40:21.437523+0000 front 2024-06-14T22:40:21.437580+0000 (oldest deadline 2024-06-14T22:40:45.537314+0000) 2024-06-14T22:40:54.776 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:54 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7[56403]: 2024-06-14T22:40:54.388+0000 7f2430fb4640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.178:6804 osd.4 since back 2024-06-14T22:40:29.638177+0000 front 2024-06-14T22:40:29.638204+0000 (oldest deadline 2024-06-14T22:40:53.738136+0000) 2024-06-14T22:40:55.945 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:55 smithi178 podman[68449]: 2024-06-14 22:40:55.615840598 +0000 UTC m=+5.416176158 container died cbc310d083f0cd1255024e890dc6a33df887e8de1789d47031c9050497e47015 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240612, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, GIT_BRANCH=HEAD, RELEASE=quincy-e47ff4b) 2024-06-14T22:40:56.275 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:56 smithi178 podman[68449]: 2024-06-14 22:40:56.088940406 +0000 UTC m=+5.889276006 container cleanup cbc310d083f0cd1255024e890dc6a33df887e8de1789d47031c9050497e47015 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7, io.buildah.version=1.36.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=quincy-e47ff4b, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240612) 2024-06-14T22:40:56.275 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:56 smithi178 bash[68449]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7 2024-06-14T22:40:56.689 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:56 smithi178 podman[68469]: 2024-06-14 22:40:56.373745329 +0000 UTC m=+0.753448317 container remove cbc310d083f0cd1255024e890dc6a33df887e8de1789d47031c9050497e47015 (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7, ceph=True, RELEASE=quincy-e47ff4b, org.label-schema.build-date=20240612, io.buildah.version=1.36.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-06-14T22:40:56.989 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:56 smithi178 podman[68566]: 2024-06-14 22:40:56.595807472 +0000 UTC m=+0.017396949 image pull 37f9eedc23c32ee6d04590e8a81233336d2c14cd278f52a2a2746a97e6ef2d95 quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580 2024-06-14T22:40:56.990 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:56 smithi178 podman[68566]: 2024-06-14 22:40:56.699369839 +0000 UTC m=+0.120959315 container create 2673621587f5c2c32eb73cbf2e59b08b483618c446553b66fadcd5258b6880fe (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7-deactivate, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240612, maintainer=Guillaume Abrioux , RELEASE=quincy-e47ff4b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.36.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-06-14T22:40:56.990 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:56 smithi178 podman[68566]: 2024-06-14 22:40:56.901673805 +0000 UTC m=+0.323263282 container init 2673621587f5c2c32eb73cbf2e59b08b483618c446553b66fadcd5258b6880fe (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7-deactivate, ceph=True, GIT_CLEAN=True, RELEASE=quincy-e47ff4b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240612) 2024-06-14T22:40:56.990 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:56 smithi178 podman[68566]: 2024-06-14 22:40:56.907342688 +0000 UTC m=+0.328932166 container start 2673621587f5c2c32eb73cbf2e59b08b483618c446553b66fadcd5258b6880fe (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240612, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, RELEASE=quincy-e47ff4b) 2024-06-14T22:40:56.990 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:56 smithi178 podman[68566]: 2024-06-14 22:40:56.993081492 +0000 UTC m=+0.414670973 container attach 2673621587f5c2c32eb73cbf2e59b08b483618c446553b66fadcd5258b6880fe (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, io.buildah.version=1.36.0, org.label-schema.build-date=20240612, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=quincy-e47ff4b) 2024-06-14T22:40:57.275 INFO:journalctl@ceph.osd.7.smithi178.stdout:Jun 14 22:40:57 smithi178 podman[68566]: 2024-06-14 22:40:57.149542255 +0000 UTC m=+0.571131745 container died 2673621587f5c2c32eb73cbf2e59b08b483618c446553b66fadcd5258b6880fe (image=quay.ceph.io/ceph-ci/ceph@sha256:7fd30dfff71893ab9541e2e03f37cd56509c88398ea6d5de47fbc7efe7a87580, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, io.buildah.version=1.36.0, org.label-schema.vendor=CentOS, GIT_COMMIT=4a8012357b057f80e264ea5a6c36c04375a85d47, org.label-schema.build-date=20240612, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=quincy-e47ff4b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-14T22:40:57.807 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@osd.7.service' 2024-06-14T22:40:57.845 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-14T22:40:57.846 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-14T22:40:57.846 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-06-14T22:40:57.846 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@prometheus.a 2024-06-14T22:40:58.173 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:40:57 smithi178 systemd[1]: Stopping Ceph prometheus.a for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:40:58.471 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:40:58 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:40:58.176Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-06-14T22:40:58.471 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:40:58 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:40:58.177Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-06-14T22:40:58.472 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:40:58 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:40:58.177Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-06-14T22:40:58.472 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:40:58 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:40:58.177Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-06-14T22:40:58.472 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:40:58 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:40:58.177Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-06-14T22:40:58.472 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:40:58 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:40:58.177Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-06-14T22:40:58.472 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:40:58 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:40:58.177Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-06-14T22:40:58.472 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:40:58 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:40:58.177Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-06-14T22:40:58.472 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:40:58 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:40:58.177Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-06-14T22:40:58.473 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:40:58 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:40:58.177Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-06-14T22:40:58.473 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:40:58 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:40:58.177Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-06-14T22:40:58.473 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:40:58 smithi178 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a[65022]: ts=2024-06-14T22:40:58.177Z caller=main.go:1147 level=info msg="See you next time!" 2024-06-14T22:40:58.473 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:40:58 smithi178 podman[68693]: 2024-06-14 22:40:58.187644715 +0000 UTC m=+0.109953355 container died cdc6069291089375981a835f2255eb57a74f00cc3aefab89b4990cf1e5c76151 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-14T22:40:58.775 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:40:58 smithi178 podman[68693]: 2024-06-14 22:40:58.657209649 +0000 UTC m=+0.579518312 container cleanup cdc6069291089375981a835f2255eb57a74f00cc3aefab89b4990cf1e5c76151 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-06-14T22:40:58.775 INFO:journalctl@ceph.prometheus.a.smithi178.stdout:Jun 14 22:40:58 smithi178 bash[68693]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-prometheus-a 2024-06-14T22:40:59.072 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@prometheus.a.service' 2024-06-14T22:40:59.106 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-14T22:40:59.106 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-06-14T22:40:59.106 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 --force --keep-logs 2024-06-14T22:41:00.943 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:41:00 smithi046 systemd[1]: Stopping Ceph alertmanager.a for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:41:01.382 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:41:01 smithi046 ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a[74477]: ts=2024-06-14T22:41:01.063Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-06-14T22:41:01.382 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:41:01 smithi046 podman[83922]: 2024-06-14 22:41:01.08435694 +0000 UTC m=+0.133657615 container died 746b52e70c667d778450660efb38a31dd12c7046dc2598cccf323a91108ac0a4 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-14T22:41:01.693 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:41:01 smithi046 podman[83922]: 2024-06-14 22:41:01.484737006 +0000 UTC m=+0.534037680 container cleanup 746b52e70c667d778450660efb38a31dd12c7046dc2598cccf323a91108ac0a4 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-14T22:41:01.693 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:41:01 smithi046 bash[83922]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a 2024-06-14T22:41:02.114 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:41:01 smithi046 podman[83941]: 2024-06-14 22:41:01.70246869 +0000 UTC m=+0.609555424 container remove 746b52e70c667d778450660efb38a31dd12c7046dc2598cccf323a91108ac0a4 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-06-14T22:41:02.114 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:41:01 smithi046 podman[83941]: 2024-06-14 22:41:01.77795005 +0000 UTC m=+0.685036748 volume remove e4762026dba5c496f544dfc64e74fa037f9168e3a4a3efb9237dbd4dca02eba4 2024-06-14T22:41:02.115 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:41:01 smithi046 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-06-14T22:41:02.115 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:41:01 smithi046 systemd[1]: Stopped Ceph alertmanager.a for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:41:02.115 INFO:journalctl@ceph.alertmanager.a.smithi046.stdout:Jun 14 22:41:01 smithi046 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@alertmanager.a.service: Consumed 1.126s CPU time. 2024-06-14T22:41:02.374 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:41:02 smithi046 systemd[1]: Stopping Ceph node-exporter.a for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:41:02.636 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:41:02 smithi046 podman[84054]: 2024-06-14 22:41:02.37800454 +0000 UTC m=+0.113593576 container died 7ccc5883403b21008de7788b9b600a6bd0447fdc692e42ea8a5c885f55787daa (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-14T22:41:02.943 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:41:02 smithi046 podman[84054]: 2024-06-14 22:41:02.73873691 +0000 UTC m=+0.474325945 container cleanup 7ccc5883403b21008de7788b9b600a6bd0447fdc692e42ea8a5c885f55787daa (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-14T22:41:02.943 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:41:02 smithi046 bash[84054]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a 2024-06-14T22:41:03.300 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:41:02 smithi046 podman[84072]: 2024-06-14 22:41:02.989826552 +0000 UTC m=+0.615607091 container remove 7ccc5883403b21008de7788b9b600a6bd0447fdc692e42ea8a5c885f55787daa (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-06-14T22:41:03.300 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:41:02 smithi046 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-06-14T22:41:03.300 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:41:03 smithi046 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-06-14T22:41:03.300 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:41:03 smithi046 systemd[1]: Stopped Ceph node-exporter.a for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:41:03.301 INFO:journalctl@ceph.node-exporter.a.smithi046.stdout:Jun 14 22:41:03 smithi046 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@node-exporter.a.service: Consumed 2.119s CPU time. 2024-06-14T22:41:11.085 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 --force --keep-logs 2024-06-14T22:41:12.775 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:41:12 smithi178 systemd[1]: Stopping Ceph grafana.a for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:41:13.190 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:41:12 smithi178 podman[69122]: 2024-06-14 22:41:12.940756712 +0000 UTC m=+0.124164901 container died 2177c25f9d220bfdf08d8e46ea7238692481e923867af5a3e1e0eb60a2418aaf (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a, maintainer=Paul Cuzner , architecture=x86_64, vcs-type=git, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.display-name=Red Hat Universal Base Image 8, name=ubi8, build-date=2022-04-29T11:48:11.055897, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=ubi8-container, summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, distribution-scope=public, io.openshift.expose-services=, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., release=239.1651231664, io.buildah.version=1.32.0, description=Ceph Grafana Container, vendor=Red Hat, Inc., io.openshift.tags=base rhel8) 2024-06-14T22:41:13.525 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:41:13 smithi178 podman[69122]: 2024-06-14 22:41:13.301881969 +0000 UTC m=+0.485290158 container cleanup 2177c25f9d220bfdf08d8e46ea7238692481e923867af5a3e1e0eb60a2418aaf (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a, summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, io.openshift.expose-services=, architecture=x86_64, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., description=Ceph Grafana Container, name=ubi8, release=239.1651231664, distribution-scope=public, io.k8s.display-name=Red Hat Universal Base Image 8, io.buildah.version=1.32.0, com.redhat.component=ubi8-container, vendor=Red Hat, Inc., io.openshift.tags=base rhel8, maintainer=Paul Cuzner , vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, vcs-type=git) 2024-06-14T22:41:13.525 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:41:13 smithi178 bash[69122]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a 2024-06-14T22:41:13.929 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:41:13 smithi178 podman[69139]: 2024-06-14 22:41:13.561371234 +0000 UTC m=+0.619421301 container remove 2177c25f9d220bfdf08d8e46ea7238692481e923867af5a3e1e0eb60a2418aaf (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-grafana-a, name=ubi8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, maintainer=Paul Cuzner , release=239.1651231664, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.32.0, architecture=x86_64, io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, vcs-type=git, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vendor=Red Hat, Inc., distribution-scope=public, io.openshift.tags=base rhel8, version=8.5, description=Ceph Grafana Container, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.openshift.expose-services=) 2024-06-14T22:41:13.929 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:41:13 smithi178 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-06-14T22:41:13.929 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:41:13 smithi178 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-06-14T22:41:13.929 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:41:13 smithi178 systemd[1]: Stopped Ceph grafana.a for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:41:13.929 INFO:journalctl@ceph.grafana.a.smithi178.stdout:Jun 14 22:41:13 smithi178 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@grafana.a.service: Consumed 4.505s CPU time. 2024-06-14T22:41:14.190 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:41:13 smithi178 systemd[1]: Stopping Ceph node-exporter.b for cb009720-2a9d-11ef-bc9d-c7b262605968... 2024-06-14T22:41:14.460 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:41:14 smithi178 podman[69253]: 2024-06-14 22:41:14.194497224 +0000 UTC m=+0.119896637 container died 03c1664adb5aed8c0e6ad5f666f5175f69fa718639019cdf029a1fc48b5cf1d3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-14T22:41:14.775 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:41:14 smithi178 podman[69253]: 2024-06-14 22:41:14.647159169 +0000 UTC m=+0.572558580 container cleanup 03c1664adb5aed8c0e6ad5f666f5175f69fa718639019cdf029a1fc48b5cf1d3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-14T22:41:14.775 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:41:14 smithi178 bash[69253]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b 2024-06-14T22:41:15.188 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:41:14 smithi178 podman[69270]: 2024-06-14 22:41:14.881585941 +0000 UTC m=+0.690966769 container remove 03c1664adb5aed8c0e6ad5f666f5175f69fa718639019cdf029a1fc48b5cf1d3 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-cb009720-2a9d-11ef-bc9d-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-06-14T22:41:15.188 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:41:14 smithi178 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-06-14T22:41:15.188 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:41:15 smithi178 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-06-14T22:41:15.188 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:41:15 smithi178 systemd[1]: Stopped Ceph node-exporter.b for cb009720-2a9d-11ef-bc9d-c7b262605968. 2024-06-14T22:41:15.188 INFO:journalctl@ceph.node-exporter.b.smithi178.stdout:Jun 14 22:41:15 smithi178 systemd[1]: ceph-cb009720-2a9d-11ef-bc9d-c7b262605968@node-exporter.b.service: Consumed 2.043s CPU time. 2024-06-14T22:41:20.663 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-14T22:41:20.697 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-14T22:41:20.724 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-14T22:41:20.726 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/crash to /home/teuthworker/archive/yuriw-2024-06-14_18:07:03-upgrade:quincy-x-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7755958/remote/smithi046/crash 2024-06-14T22:41:20.727 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/crash -- . 2024-06-14T22:41:20.786 INFO:teuthology.orchestra.run.smithi046.stderr:tar: /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/crash: Cannot open: No such file or directory 2024-06-14T22:41:20.786 INFO:teuthology.orchestra.run.smithi046.stderr:tar: Error is not recoverable: exiting now 2024-06-14T22:41:20.787 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/crash to /home/teuthworker/archive/yuriw-2024-06-14_18:07:03-upgrade:quincy-x-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7755958/remote/smithi178/crash 2024-06-14T22:41:20.788 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/crash -- . 2024-06-14T22:41:20.827 INFO:teuthology.orchestra.run.smithi178.stderr:tar: /var/lib/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/crash: Cannot open: No such file or directory 2024-06-14T22:41:20.827 INFO:teuthology.orchestra.run.smithi178.stderr:tar: Error is not recoverable: exiting now 2024-06-14T22:41:20.828 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-14T22:41:20.828 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-06-14T22:41:20.858 INFO:teuthology.orchestra.run.smithi046.stderr:grep: /var/log/ceph/cb009720-2a9d-11ef-bc9d-c7b262605968/ceph.log: No such file or directory 2024-06-14T22:41:20.860 INFO:tasks.cephadm:Compressing logs... 2024-06-14T22:41:20.860 DEBUG:teuthology.orchestra.run.smithi046:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-14T22:41:20.903 DEBUG:teuthology.orchestra.run.smithi178:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-14T22:41:20.926 INFO:teuthology.orchestra.run.smithi178.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-14T22:41:20.929 INFO:teuthology.orchestra.run.smithi046.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-14T22:41:22.018 INFO:tasks.cephadm:Archiving logs... 2024-06-14T22:41:22.018 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-06-14_18:07:03-upgrade:quincy-x-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7755958/remote/smithi046/log 2024-06-14T22:41:22.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-14T22:41:22.227 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-06-14_18:07:03-upgrade:quincy-x-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7755958/remote/smithi178/log 2024-06-14T22:41:22.228 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-14T22:41:22.327 INFO:tasks.cephadm:Removing cluster... 2024-06-14T22:41:22.327 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 --force 2024-06-14T22:41:22.817 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid cb009720-2a9d-11ef-bc9d-c7b262605968 --force 2024-06-14T22:41:23.292 INFO:tasks.cephadm:Removing cephadm ... 2024-06-14T22:41:23.293 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-14T22:41:23.311 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-14T22:41:23.326 INFO:tasks.cephadm:Teardown complete 2024-06-14T22:41:23.326 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-14T22:41:23.337 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-14T22:41:23.337 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-14T22:41:23.353 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-14T22:41:23.443 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-14T22:41:23.453 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-14T22:41:23.453 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-14T22:41:23.456 DEBUG:teuthology.orchestra.run.smithi178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-14T22:41:23.470 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-06-14T22:41:23.471 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-06-14T22:41:23.475 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-14T22:41:23.475 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-06-14T22:41:23.475 INFO:teuthology.orchestra.run.smithi178.stdout:^* hv01.front.sepia.ceph.com 3 6 377 33 +1023us[ +985us] +/- 50ms 2024-06-14T22:41:23.476 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 -776us[ -776us] +/- 50ms 2024-06-14T22:41:23.476 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 -694us[ -694us] +/- 66ms 2024-06-14T22:41:23.476 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-06-14T22:41:23.477 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-14T22:41:23.477 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-06-14T22:41:23.477 INFO:teuthology.orchestra.run.smithi046.stdout:^* hv01.front.sepia.ceph.com 3 6 377 34 +1042us[+1081us] +/- 50ms 2024-06-14T22:41:23.477 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 31 -779us[ -779us] +/- 50ms 2024-06-14T22:41:23.477 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 -707us[ -668us] +/- 66ms 2024-06-14T22:41:23.478 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-06-14T22:41:23.478 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-14T22:41:23.487 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-14T22:41:23.488 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-14T22:41:23.519 DEBUG:teuthology.orchestra.run.smithi046:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-14T22:41:23.605 DEBUG:teuthology.orchestra.run.smithi178:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-14T22:41:23.674 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-06-14T22:41:23.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-14T22:41:23.706 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-06-14T22:41:23.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-14T22:41:23.735 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-14T22:41:23.744 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-14T22:41:23.769 INFO:teuthology.task.internal:Duration was 1017.202817 seconds 2024-06-14T22:41:23.770 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-14T22:41:23.811 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-14T22:41:23.812 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-14T22:41:23.814 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-14T22:41:23.852 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-14T22:41:23.853 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-14T22:41:24.243 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-14T22:41:24.244 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2024-06-14T22:41:24.244 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-14T22:41:24.306 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi178.front.sepia.ceph.com 2024-06-14T22:41:24.307 DEBUG:teuthology.orchestra.run.smithi178:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-14T22:41:24.328 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-14T22:41:24.329 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-14T22:41:24.349 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-14T22:41:24.394 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-14T22:41:24.395 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-14T22:41:24.418 DEBUG:teuthology.orchestra.run.smithi178:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-14T22:41:24.687 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-14T22:41:24.697 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-14T22:41:24.697 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-14T22:41:24.723 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-14T22:41:24.748 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-14T22:41:24.757 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-14T22:41:24.765 DEBUG:teuthology.orchestra.run.smithi178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-14T22:41:24.786 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2024-06-14T22:41:24.810 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = core 2024-06-14T22:41:24.832 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-14T22:41:24.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-14T22:41:24.859 DEBUG:teuthology.orchestra.run.smithi178:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-14T22:41:24.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-14T22:41:24.886 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-14T22:41:24.896 INFO:teuthology.task.internal:Transferring archived files... 2024-06-14T22:41:24.897 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-06-14_18:07:03-upgrade:quincy-x-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7755958/remote/smithi046 2024-06-14T22:41:24.897 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-14T22:41:24.939 DEBUG:teuthology.misc:Transferring archived files from smithi178:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-06-14_18:07:03-upgrade:quincy-x-wip-yuri5-testing-2024-06-08-0902-reef-distro-default-smithi/7755958/remote/smithi178 2024-06-14T22:41:24.939 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-14T22:41:24.974 INFO:teuthology.task.internal:Removing archive directory... 2024-06-14T22:41:24.975 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-14T22:41:24.977 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-14T22:41:25.024 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-14T22:41:25.036 INFO:teuthology.task.internal:Not uploading archives. 2024-06-14T22:41:25.036 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-14T22:41:25.071 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-14T22:41:25.071 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-14T22:41:25.073 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-14T22:41:25.086 INFO:teuthology.orchestra.run.smithi046.stdout: 262165 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jun 14 22:41 /home/ubuntu/cephtest 2024-06-14T22:41:25.086 INFO:teuthology.orchestra.run.smithi046.stdout: 531222 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Jun 14 22:39 /home/ubuntu/cephtest/mnt.0 2024-06-14T22:41:25.087 INFO:teuthology.orchestra.run.smithi046.stdout: 531223 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 14 22:39 /home/ubuntu/cephtest/mnt.0/client.0 2024-06-14T22:41:25.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-14T22:41:25.088 INFO:teuthology.orchestra.run.smithi046.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-06-14T22:41:25.088 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_c1b06c2b5d3a02a3583265a115f09e052855bcf2/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi046 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-06-14T22:41:25.095 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-14T22:41:25.106 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-14T22:41:25.139 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on smithi046 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-06-14T22:41:25.142 INFO:teuthology.run:Summary data: description: upgrade:quincy-x/stress-split/{0-distro/centos_9.stream_runc 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/classic} duration: 1017.2028167247772 failure_reason: 'Command failed on smithi046 with status 128: ''rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone --depth 1 --branch quincy https://github.com/cbodley/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0''' flavor: default owner: scheduled_yuriw@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=602b20f258fd42c79b1ea2ff8e2a8c60 status: fail success: false 2024-06-14T22:41:25.142 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-14T22:41:25.149 INFO:teuthology.orchestra.run.smithi178.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 14 22:41 /home/ubuntu/cephtest 2024-06-14T22:41:25.229 INFO:teuthology.run:FAIL